3 ms·

Strange File Resizing on DOS

38 points by ingve 2y ago

RecycledEle 2y ago
I think I remember being told there was no way to shrink files when I was studying computer programming in college. I was taught to copy the now shorter file into a new file then delete the original file.
jsjohnst 2y ago
Hadn’t heard this tidbit before, but it’s definitely on brand for most other things developed in the early Microsoft era. Significant portions were heavily inspired by prior work in most cases, with some outright copied or acquired from the original source.
PreInternet01 2y ago
Yeah, "in order to create a file of N MB, just seek to the offset you need and write 0 bytes" has been one of my favorite pieces of trivia for a long while -- this pattern even survives in Win32's SetFilePointer. On Windows, the resulting file is guaranteed to have any intervening bytes zero-initialized, but for DOS, that wasn't always the case, and any thus-created file could recycle previously deleted data, especially on redirected filesystems (e.g. LAN Manager or Novell NetWare volumes).
- rep_lodsb 2y ago
  I don't think DOS ever wrote zeroes when extending a file (except when using the FCB function "write random with zero fill"). On a single-user system, you'd only get back your own deleted data, so this at least can't be considered a security problem. It definitely is a problem if a network server extends files that way! NTFS has a "zero fill" flag that the OS sets for newly allocated blocks, so it doesn't actually have to fill them on disk - it can just return zeroes when an application tries to read from a part of the file that hasn't been written yet. I believe ext4 and all other modern filesystems have this optimization as well.
  - diggernet 2y ago
    It's only not a security problem if the file never leaves that single user system. Which was definitely not a safe assumption, even then.
  - PreInternet01 2y ago
    > you'd only get back your own deleted data That depends: if the operation was on a floppy you got from someone else, that data might very well be theirs. > NTFS has a "zero fill" flag True, but irrelevant: the Windows API guarantees the "seeking to an offset greater than the file length and then writing anything, including nothing, will fill the intervening bytes with zeroes" behavior. How this translates to the underlying file system (which may very well not be NTFS) is an implementation detail.
stevekemp 2y ago
The author writes "CP/M 2.2 had no mechanism for resizing files", but that is not exactly true. CP/M had a submit-command, which would read a text-file containing a list of commands and write out a file named $$$.SUB to the A:. Later when the CP/M booted (via a cold start, or a warm start which was triggered by pressing Ctrl-C) it would be processed. The $$$.SUB file was processed in reverse, and it would be assumed to contain N entries of 128 bytes each. The last record would be read, the command be executed, and the file would be resized down by 128 bytes. So given input like: FOO BAR BAZ The CP/M command-processor (i.e. shell) would run "BAZ", truncate the $$$.SUB file by 128 bytes. Then it would do the same, executing "BAR", truncating down, and then running "FOO", truncating down to zero, and erasing. How do I know this? I wrote a CP/M emulator and had a devil of a time working out the behaviour, but the CP/M "Close File" call will truncate a file is some values of the FCB tell it to. Since CP/M itself used this I can only assume that user-programs did too.
- rep_lodsb 2y ago
  "Close file" will update the directory entry for the current extent. It doesn't (AFAIK) delete other extents or free any blocks that have been allocated to the file. The "last record" field of the FCB / direntry will be decremented for each line, so that the shell can keep track of where in the file it should continue (this can't be stored in memory, because programs are explicitly allowed to overwrite the command processor). But the space on disk won't be freed until it's done executing all commands in the file. For an emulator it makes sense to actually truncate the file, because on other systems there is likely no way to store a "logical" file size that is shorter than the actual one.

38 points by ingve 2y ago

I think I remember being told there was no way to shrink files when I was studying computer programming in college. I was taught to copy the now shorter file into a new file then delete the original file.

jsjohnst 2y ago

Hadn’t heard this tidbit before, but it’s definitely on brand for most other things developed in the early Microsoft era. Significant portions were heavily inspired by prior work in most cases, with some outright copied or acquired from the original source.

PreInternet01 2y ago

Yeah, "in order to create a file of N MB, just seek to the offset you need and write 0 bytes" has been one of my favorite pieces of trivia for a long while -- this pattern even survives in Win32's SetFilePointer. On Windows, the resulting file is guaranteed to have any intervening bytes zero-initialized, but for DOS, that wasn't always the case, and any thus-created file could recycle previously deleted data, especially on redirected filesystems (e.g. LAN Manager or Novell NetWare volumes).

rep_lodsb 2y ago
I don't think DOS ever wrote zeroes when extending a file (except when using the FCB function "write random with zero fill"). On a single-user system, you'd only get back your own deleted data, so this at least can't be considered a security problem. It definitely is a problem if a network server extends files that way! NTFS has a "zero fill" flag that the OS sets for newly allocated blocks, so it doesn't actually have to fill them on disk - it can just return zeroes when an application tries to read from a part of the file that hasn't been written yet. I believe ext4 and all other modern filesystems have this optimization as well.
- diggernet 2y ago
  It's only not a security problem if the file never leaves that single user system. Which was definitely not a safe assumption, even then.
- PreInternet01 2y ago
  > you'd only get back your own deleted data That depends: if the operation was on a floppy you got from someone else, that data might very well be theirs. > NTFS has a "zero fill" flag True, but irrelevant: the Windows API guarantees the "seeking to an offset greater than the file length and then writing anything, including nothing, will fill the intervening bytes with zeroes" behavior. How this translates to the underlying file system (which may very well not be NTFS) is an implementation detail.

stevekemp 2y ago

The author writes "CP/M 2.2 had no mechanism for resizing files", but that is not exactly true. CP/M had a submit-command, which would read a text-file containing a list of commands and write out a file named $$$.SUB to the A:. Later when the CP/M booted (via a cold start, or a warm start which was triggered by pressing Ctrl-C) it would be processed. The $$$.SUB file was processed in reverse, and it would be assumed to contain N entries of 128 bytes each. The last record would be read, the command be executed, and the file would be resized down by 128 bytes. So given input like: FOO BAR BAZ The CP/M command-processor (i.e. shell) would run "BAZ", truncate the $$$.SUB file by 128 bytes. Then it would do the same, executing "BAR", truncating down, and then running "FOO", truncating down to zero, and erasing. How do I know this? I wrote a CP/M emulator and had a devil of a time working out the behaviour, but the CP/M "Close File" call will truncate a file is some values of the FCB tell it to. Since CP/M itself used this I can only assume that user-programs did too.

rep_lodsb 2y ago
"Close file" will update the directory entry for the current extent. It doesn't (AFAIK) delete other extents or free any blocks that have been allocated to the file. The "last record" field of the FCB / direntry will be decremented for each line, so that the shell can keep track of where in the file it should continue (this can't be stored in memory, because programs are explicitly allowed to overwrite the command processor). But the space on disk won't be freed until it's done executing all commands in the file. For an emulator it makes sense to actually truncate the file, because on other systems there is likely no way to store a "logical" file size that is shorter than the actual one.