6 ms·

An Inside Look at MS-DOS – The design decisions behind the popular OS (1983)

156 points by Lammy 4y ago

deleted 4y ago
[deleted]
Lammy 4y ago
ksaj 4y ago
I studied MS-DOS inside and out (quite literally). One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point, called the Program Segment Prefix (PSP). It is why you could simply execute a RET (return) and have it drop back to DOS again. The first bytes of the PSP are a literal INT 20h which is the Drop To DOS interrupt, and a RET with nothing left on the stack will return you to that zero offset, and then Drop to DOS. Probably a bit hacky, but really clever. I wrote my own DOS clone in Assembler, that I called PoverDOS because it was surely going to make me about that much money, and managed to make a boot sector that was far more functional than the one that comes standard with MS and PC DOS. The main thing it did was self-diagnostics to determine if the system had been infected by a boot sector virus. That was quite a feat, but became a moot point once 64 bit computing came about, because the boot sequence also gained a whole lot more space to work in. Don't bring it up with me at a party, 'cos all those DOS intricacies are still among my favourite subjects, and 97% of the population probably would not be nearly as enthusiastic. DOS is still the party killer it always was.
- Dracophoenix 4y ago
  How much of DOS did you learn by reading MS's raw source code vs reversing the OS yourself?
  - ksaj 4y ago
    I glanced through it. I didn't learn anything new, except the definition of bloat. The raw source wasn't available when I was doing this, so I learned it entirely by reverse engineering, and documentation. Ironically I learned how the boot sector worked and basic ASM purely by using DEBUG.EXE. My computer got infected by Stoned virus, and I was pretty excited about it! This was back when DOS came with a green binder with actual documentation. It was a year or so of that before I discovered TASM and then MASM assemblers, which of course are a LOT more human-friendly. The earlier versions of DOS were entirely written in ASM, and they were super-tight. But later versions were all rewritten in high level languages. The stuff in the leak is all the bloated high-level language versions.
- Aloha 4y ago
  You and I would have good party conversations
  - fuzzfactor 4y ago
    Well tonight we're going to party like it's 1999. This article was originally published right before the issuance of MS-DOS 2.0. That was not even near the '90's yet. By the time this was captured in the year 2000, the MSDOS.SYS file had already become almost a dummy file of few bytes in Win9x. The DOS OS itself with only the native commands is basically contained within the IO.SYS, MSDOS.SYS, and COMMAND.COM files, originally written in that order starting at the first working sector of a formatted floppy disk so it would boot. On a hard drive the 3 files would have to be written starting at the first working formatted sector of an actively flagged primary partition, and a proper master boot record needs to be on sector 0 (along with a valid partition table) if any boot files on that HDD are to be recognized by BIOS to begin with. Otherwise it would not boot. IOW once the volume boots you can do things like copy files and list directories, but if you want to edit files or format other volumes you needed more than just the 3 OS files. You needed the additional OS apps like EDIT.COM and FORMAT.COM on your floppy so they would be there when you needed them after you booted. Once booted, the SYS command could be used to properly write the OS to a target volume, so you needed SYS.COM for that. And by W9x, the three DOS OS files would no longer need to be right there at the start of the volume (as if they were the first things copied to the volume), and DOS FDISK would handle HDDs of hundreds of GB even though each FAT32 partition was only supported up to 32GB. The undocumented FDISK /MBR command executed from an A: or B: (floppy) prompt would simply write a fresh bootable DOS master boot record to sector 0 of C: (what should usually be considered) your only HDD on the mainboard. Without disturbing the partition table there. The SYS command also replaces the Volume Boot Record (bootsector) of the target volume with a fresh DOS bootsector as it freshly (over)writes IO.SYS, MSDOS.SYS, and COMMAND.COM into the file system. So you want to make a zeroed HDD boot, first boot to a DOS floppy (USB stick will do): Run FDISK and create a primary partition (bigger than 512MB) smaller than 32GB on the fixed (HDD) disk. Make the partition Active. (Alternatively you could use some other way to make a suitable partition, like Linux or a modern Windows version but the resulting "geometry" of the disk layout may not be the same or fully recognizable by DOS, which can affect the bootability. Sometimes with Linux or Windows, the older the version the better. Outcomes can also be surprisingly interesting when certain sectors of the HDD are not exactly zeroed beforehand. AMD also not quite the same as Intel, nor old Seagate identical to WD.) Now you've got a partition table on sector 0 with one primary partition marked "active" so the BIOS will boot to it. Run FDISK /MBR and it silently writes an MBR to sector 0 so the BIOS will REALLY boot to it. (You may need to manually copy a working MBR to the first 440 bytes of sector 0 if the FDISK /MBR command does not do the trick. The MBR is supposed to be generic for DOS and a Linux version like MBR.bin from the Syslinux project can be copied bytewise with binary tools. Newer Windows will also upgrade the MBR but it will still boot DOS (or Linux) afterward from a submenu.) All other sectors are still unchanged. Run FORMAT on the partition and I would always suggest labeling it as to HDD/Partition# when done. 11 characters max. Now you've got something other than zeros in places other than sector 0 of your HDD, but still no data actually located on the formatted volume yet. Run SYS on the formatted volume and it properly writes the 3 critical DOS OS files to the volume. Power down, remove the floppy or USB device. The HDD should then boot DOS according to mainboard BIOS priority. Add W9x to the same FAT32 volume and you still get a DOS boot entry on your Microsoft boot menu for when you want plain DOS. Install WXP to a different formatted partition (even FAT32) and your old W9x/DOS volume still can boot when desired as a submenu of the NT5 NTloader that replaces the DOS bootloader on that active partition. Then with Win10 (non-UEFI setup) on a third (NTFS) partition you can have an NTLDR submenu within the replacement NT6 Bootmgr and boot any of the above. Note the volume you installed the new Windows version to does not become active itself nor contain any boot files. By default additional boot entries are made to the established active partition. Even logical partitions containing OS's can be booted this way, according to OS version, from one of the few primary partitions you are allowed to flag as active. BIOS only. Too bad with foolish-type UEFI or GPT, no DOS for you. Also remember with bare-metal DOS/W9x and audio/MIDI (no USB), no latency for you either. Even on dog-slow PC's. Things like that.
- ipython 4y ago
  Awesome! I encourage you to post your code and blog about it. I would find it fascinating, and suspect others would too.
  - ksaj 4y ago
    I didn't write a blog about it, but the code for the boot sector and installer (dropper) is here: https://github.com/ksaj/antivirus-mbr https://github.com/ksaj/antivirus-mbr I separated it out because it was useful for all versions of DOS at the time, and boot sectors, being outside the DOS realm, were always a sitting duck. I documented it to death, so it is super easy to read even though it is ASM code.
- zozbot234 4y ago
  > One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point ... AIUI that design was in CP/M already, and retained for compatibility.
  - rzzzt 4y ago
    The two File Control Blocks are also vestigial remains of CP/M, and they can also be found in the same section of memory.
- hutzlibu 4y ago
  "and 97% of the population probably would not be nearly as enthusiastic" Thats likely a strong understatement.
- mysterydip 4y ago
  Did you ever write a book? I'd love to read up on this kind of stuff.
  - hyperman1 4y ago
    Other people did e.g Peter Norton wrote a whole library around this stuff.
- stevekemp 4y ago
  For me my reversing started and ending with discovering and documenting the List of Lists: https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html That allowed TSRs to do clever things. I didn't use CP/M before DOS, but these days I know all about the PSP via writing code for CP/M, in Z80. Kinda impressive how long all these backwards compatibility things last.
  - mickeyp 4y ago
    One of the first things I looked up, as a budding Turbo C developer in the DOS era, as it held all manner of secrets. I seem to recall 'one guy' had this hand-curated master list of interrupts. Might've been this list; perhaps another. I was obsessed with TSR programs back then. The notion that you can run a background process (in exchange for gobbling up all-valuable conventional memory) thanks to clever hacking really captured my eye back then.
    - mrlonglong 4y ago
      Chap you're referring to is Ralf Brown, he published a list of all known interrupts. It is on the web https://www.cs.cmu.edu/~ralf/files.html https://www.cs.cmu.edu/~ralf/files.html
      - ksaj 4y ago
        There was a pair of books that were instrumental to me: Undocumented PC and Undocumented DOS. Those books taught me so many tricks! I still remember the DWIM (Do what I mean) interrupt they used as the demonstration of how each interrupt was documented.
        rasz 4y ago
        The Undocumented PC, A Programmer's Guide to I/O, CPUs, and Fixed Memory Areas by Frank van Gilluwe is great. Frank van Gilluwe is the author of Sourcer disassembler https://corexor.wordpress.com/2015/12/09/sourcer-and-windows-source/ https://corexor.wordpress.com/2015/12/09/sourcer-and-windows...
dang 4y ago
We've changed the URL from https://web.archive.org/web/20001003150639/http://www.patersontech.com/Dos/Byte/InsideDos.htm https://web.archive.org/web/20001003150639/http://www.paters... to what looks like the current canonical source. Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html (It's fine to post archive.org links if there's really no other way, but please search for another way first—there are quite a few editions of this article on the web, which you can find by googling sentences from the text—and try to find the canonical source if possible!) It's a great submission otherwise! and appears never to have been discussed on HN before, which is an unusual find.
- rkagerer 4y ago
  Thanks for denoting the initial archive.org link here, the text is identical and the lack of ads make it an easier read.
  - dang 4y ago
    Archive links are always welcome in the comments! just best to avoid as the primary URL, unless there's no other option.
- Lammy 4y ago
  Sorry, site was/is not loading for me so I thought it was gone :)
pavlov 4y ago
This commentary on MS-DOS 1.x was written by the operating system’s original author, Tim Paterson. His company Seattle Computer Products sold the source code to Microsoft for $50k in what is often regarded as one of Bill Gates’s greatest acquisitions. He had been approached by IBM to provide an OS for their upcoming Personal Computer but didn’t actually have a product. Of course he didn’t tell this to SCP when making the licensing deal. Microsoft’s contract specified a per-customer fee paid to SCP, but they really had only one customer in mind — the largest computer company in the world. Although it seems to me that Microsoft already had enough talented engineers in 1981 that they probably could have written a CP/M clone from scratch? But maybe the timeline they promised IBM was too tight, in which case Paterson / SCP was indeed the perfect unwitting savior. (When these events took place I was wearing diapers, so it’s all third-hand information. The great thing about HN is that there’s probably someone around who was in the room for half of it, and then somebody else who witnessed the other side.)
- predictsoft 4y ago
  Of course even though SCP sold for 50k, Tim ended up working for Microsoft, so his stock options likely made him wealthy.
regular 4y ago
voidz 4y ago
The question I've latently had for decades: "what's the difference between .COM and .EXE", is answered by this article. Great reading too, brings back some good memories about MS-DOS tinkering. customizing the prompt (prompt $p$g), fdisk /mbr, sys command that sets up boot files (msdos.sys io.sys and command.com). And oh the time I have spent to write menu entries in autoexec.bat and config.sys, figuring everything out from floppy disks containing things like cdrom drivers. I wasn't much older than 10 years old back then and was so happy when someone gave me an update: MS-DOS 6.22.
29athrowaway 4y ago
I preferred DR-DOS, but Microsoft and its anti-competitive initiatives killed it. The author of this article is the guy who invented the original DOS, which was acquired by Microsoft for peanuts. If Seattle Computer Products had sold their product to IBM directly under the same terms they would be as rich as Bill Gates today.
mrlonglong 4y ago
DOS should be able to boot in UEFI with a hack, write an UEFi stub that loads and set up the interrupt vectors and switch into real mode before loading the rest of the MS-DOS boot files.
- toast0 4y ago
  Video might be a problem. I think intel integrated graphics might be UEFI only now, I don't know if they still have the text modes that DOS uses.
  - mrlonglong 4y ago
    Quite,you set up the interrupt vectors and mask off the video memory regions so access throws an exception and all this can be handled in the exception handlers

156 points by Lammy 4y ago

deleted 4y ago

[deleted]

Lammy 4y ago

ksaj 4y ago

I studied MS-DOS inside and out (quite literally). One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point, called the Program Segment Prefix (PSP). It is why you could simply execute a RET (return) and have it drop back to DOS again. The first bytes of the PSP are a literal INT 20h which is the Drop To DOS interrupt, and a RET with nothing left on the stack will return you to that zero offset, and then Drop to DOS. Probably a bit hacky, but really clever. I wrote my own DOS clone in Assembler, that I called PoverDOS because it was surely going to make me about that much money, and managed to make a boot sector that was far more functional than the one that comes standard with MS and PC DOS. The main thing it did was self-diagnostics to determine if the system had been infected by a boot sector virus. That was quite a feat, but became a moot point once 64 bit computing came about, because the boot sequence also gained a whole lot more space to work in. Don't bring it up with me at a party, 'cos all those DOS intricacies are still among my favourite subjects, and 97% of the population probably would not be nearly as enthusiastic. DOS is still the party killer it always was.

Dracophoenix 4y ago
How much of DOS did you learn by reading MS's raw source code vs reversing the OS yourself?
- ksaj 4y ago
  I glanced through it. I didn't learn anything new, except the definition of bloat. The raw source wasn't available when I was doing this, so I learned it entirely by reverse engineering, and documentation. Ironically I learned how the boot sector worked and basic ASM purely by using DEBUG.EXE. My computer got infected by Stoned virus, and I was pretty excited about it! This was back when DOS came with a green binder with actual documentation. It was a year or so of that before I discovered TASM and then MASM assemblers, which of course are a LOT more human-friendly. The earlier versions of DOS were entirely written in ASM, and they were super-tight. But later versions were all rewritten in high level languages. The stuff in the leak is all the bloated high-level language versions.
Aloha 4y ago
You and I would have good party conversations
- fuzzfactor 4y ago
  Well tonight we're going to party like it's 1999. This article was originally published right before the issuance of MS-DOS 2.0. That was not even near the '90's yet. By the time this was captured in the year 2000, the MSDOS.SYS file had already become almost a dummy file of few bytes in Win9x. The DOS OS itself with only the native commands is basically contained within the IO.SYS, MSDOS.SYS, and COMMAND.COM files, originally written in that order starting at the first working sector of a formatted floppy disk so it would boot. On a hard drive the 3 files would have to be written starting at the first working formatted sector of an actively flagged primary partition, and a proper master boot record needs to be on sector 0 (along with a valid partition table) if any boot files on that HDD are to be recognized by BIOS to begin with. Otherwise it would not boot. IOW once the volume boots you can do things like copy files and list directories, but if you want to edit files or format other volumes you needed more than just the 3 OS files. You needed the additional OS apps like EDIT.COM and FORMAT.COM on your floppy so they would be there when you needed them after you booted. Once booted, the SYS command could be used to properly write the OS to a target volume, so you needed SYS.COM for that. And by W9x, the three DOS OS files would no longer need to be right there at the start of the volume (as if they were the first things copied to the volume), and DOS FDISK would handle HDDs of hundreds of GB even though each FAT32 partition was only supported up to 32GB. The undocumented FDISK /MBR command executed from an A: or B: (floppy) prompt would simply write a fresh bootable DOS master boot record to sector 0 of C: (what should usually be considered) your only HDD on the mainboard. Without disturbing the partition table there. The SYS command also replaces the Volume Boot Record (bootsector) of the target volume with a fresh DOS bootsector as it freshly (over)writes IO.SYS, MSDOS.SYS, and COMMAND.COM into the file system. So you want to make a zeroed HDD boot, first boot to a DOS floppy (USB stick will do): Run FDISK and create a primary partition (bigger than 512MB) smaller than 32GB on the fixed (HDD) disk. Make the partition Active. (Alternatively you could use some other way to make a suitable partition, like Linux or a modern Windows version but the resulting "geometry" of the disk layout may not be the same or fully recognizable by DOS, which can affect the bootability. Sometimes with Linux or Windows, the older the version the better. Outcomes can also be surprisingly interesting when certain sectors of the HDD are not exactly zeroed beforehand. AMD also not quite the same as Intel, nor old Seagate identical to WD.) Now you've got a partition table on sector 0 with one primary partition marked "active" so the BIOS will boot to it. Run FDISK /MBR and it silently writes an MBR to sector 0 so the BIOS will REALLY boot to it. (You may need to manually copy a working MBR to the first 440 bytes of sector 0 if the FDISK /MBR command does not do the trick. The MBR is supposed to be generic for DOS and a Linux version like MBR.bin from the Syslinux project can be copied bytewise with binary tools. Newer Windows will also upgrade the MBR but it will still boot DOS (or Linux) afterward from a submenu.) All other sectors are still unchanged. Run FORMAT on the partition and I would always suggest labeling it as to HDD/Partition# when done. 11 characters max. Now you've got something other than zeros in places other than sector 0 of your HDD, but still no data actually located on the formatted volume yet. Run SYS on the formatted volume and it properly writes the 3 critical DOS OS files to the volume. Power down, remove the floppy or USB device. The HDD should then boot DOS according to mainboard BIOS priority. Add W9x to the same FAT32 volume and you still get a DOS boot entry on your Microsoft boot menu for when you want plain DOS. Install WXP to a different formatted partition (even FAT32) and your old W9x/DOS volume still can boot when desired as a submenu of the NT5 NTloader that replaces the DOS bootloader on that active partition. Then with Win10 (non-UEFI setup) on a third (NTFS) partition you can have an NTLDR submenu within the replacement NT6 Bootmgr and boot any of the above. Note the volume you installed the new Windows version to does not become active itself nor contain any boot files. By default additional boot entries are made to the established active partition. Even logical partitions containing OS's can be booted this way, according to OS version, from one of the few primary partitions you are allowed to flag as active. BIOS only. Too bad with foolish-type UEFI or GPT, no DOS for you. Also remember with bare-metal DOS/W9x and audio/MIDI (no USB), no latency for you either. Even on dog-slow PC's. Things like that.
ipython 4y ago
Awesome! I encourage you to post your code and blog about it. I would find it fascinating, and suspect others would too.
- ksaj 4y ago
  I didn't write a blog about it, but the code for the boot sector and installer (dropper) is here: https://github.com/ksaj/antivirus-mbr https://github.com/ksaj/antivirus-mbr I separated it out because it was useful for all versions of DOS at the time, and boot sectors, being outside the DOS realm, were always a sitting duck. I documented it to death, so it is super easy to read even though it is ASM code.
zozbot234 4y ago
> One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point ... AIUI that design was in CP/M already, and retained for compatibility.
- rzzzt 4y ago
  The two File Control Blocks are also vestigial remains of CP/M, and they can also be found in the same section of memory.
hutzlibu 4y ago
"and 97% of the population probably would not be nearly as enthusiastic" Thats likely a strong understatement.
mysterydip 4y ago
Did you ever write a book? I'd love to read up on this kind of stuff.
- hyperman1 4y ago
  Other people did e.g Peter Norton wrote a whole library around this stuff.
stevekemp 4y ago
For me my reversing started and ending with discovering and documenting the List of Lists: https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html That allowed TSRs to do clever things. I didn't use CP/M before DOS, but these days I know all about the PSP via writing code for CP/M, in Z80. Kinda impressive how long all these backwards compatibility things last.
- mickeyp 4y ago
  One of the first things I looked up, as a budding Turbo C developer in the DOS era, as it held all manner of secrets. I seem to recall 'one guy' had this hand-curated master list of interrupts. Might've been this list; perhaps another. I was obsessed with TSR programs back then. The notion that you can run a background process (in exchange for gobbling up all-valuable conventional memory) thanks to clever hacking really captured my eye back then.
  - mrlonglong 4y ago
    Chap you're referring to is Ralf Brown, he published a list of all known interrupts. It is on the web https://www.cs.cmu.edu/~ralf/files.html https://www.cs.cmu.edu/~ralf/files.html
    - ksaj 4y ago
      There was a pair of books that were instrumental to me: Undocumented PC and Undocumented DOS. Those books taught me so many tricks! I still remember the DWIM (Do what I mean) interrupt they used as the demonstration of how each interrupt was documented.
      - rasz 4y ago
        The Undocumented PC, A Programmer's Guide to I/O, CPUs, and Fixed Memory Areas by Frank van Gilluwe is great. Frank van Gilluwe is the author of Sourcer disassembler https://corexor.wordpress.com/2015/12/09/sourcer-and-windows-source/ https://corexor.wordpress.com/2015/12/09/sourcer-and-windows...

dang 4y ago

We've changed the URL from https://web.archive.org/web/20001003150639/http://www.patersontech.com/Dos/Byte/InsideDos.htm https://web.archive.org/web/20001003150639/http://www.paters... to what looks like the current canonical source. Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html (It's fine to post archive.org links if there's really no other way, but please search for another way first—there are quite a few editions of this article on the web, which you can find by googling sentences from the text—and try to find the canonical source if possible!) It's a great submission otherwise! and appears never to have been discussed on HN before, which is an unusual find.

rkagerer 4y ago
Thanks for denoting the initial archive.org link here, the text is identical and the lack of ads make it an easier read.
- dang 4y ago
  Archive links are always welcome in the comments! just best to avoid as the primary URL, unless there's no other option.
Lammy 4y ago
Sorry, site was/is not loading for me so I thought it was gone :)

pavlov 4y ago

This commentary on MS-DOS 1.x was written by the operating system’s original author, Tim Paterson. His company Seattle Computer Products sold the source code to Microsoft for $50k in what is often regarded as one of Bill Gates’s greatest acquisitions. He had been approached by IBM to provide an OS for their upcoming Personal Computer but didn’t actually have a product. Of course he didn’t tell this to SCP when making the licensing deal. Microsoft’s contract specified a per-customer fee paid to SCP, but they really had only one customer in mind — the largest computer company in the world. Although it seems to me that Microsoft already had enough talented engineers in 1981 that they probably could have written a CP/M clone from scratch? But maybe the timeline they promised IBM was too tight, in which case Paterson / SCP was indeed the perfect unwitting savior. (When these events took place I was wearing diapers, so it’s all third-hand information. The great thing about HN is that there’s probably someone around who was in the room for half of it, and then somebody else who witnessed the other side.)

predictsoft 4y ago
Of course even though SCP sold for 50k, Tim ended up working for Microsoft, so his stock options likely made him wealthy.

regular 4y ago

voidz 4y ago

The question I've latently had for decades: "what's the difference between .COM and .EXE", is answered by this article. Great reading too, brings back some good memories about MS-DOS tinkering. customizing the prompt (prompt $p$g), fdisk /mbr, sys command that sets up boot files (msdos.sys io.sys and command.com). And oh the time I have spent to write menu entries in autoexec.bat and config.sys, figuring everything out from floppy disks containing things like cdrom drivers. I wasn't much older than 10 years old back then and was so happy when someone gave me an update: MS-DOS 6.22.

29athrowaway 4y ago

I preferred DR-DOS, but Microsoft and its anti-competitive initiatives killed it. The author of this article is the guy who invented the original DOS, which was acquired by Microsoft for peanuts. If Seattle Computer Products had sold their product to IBM directly under the same terms they would be as rich as Bill Gates today.

mrlonglong 4y ago

DOS should be able to boot in UEFI with a hack, write an UEFi stub that loads and set up the interrupt vectors and switch into real mode before loading the rest of the MS-DOS boot files.

toast0 4y ago
Video might be a problem. I think intel integrated graphics might be UEFI only now, I don't know if they still have the text modes that DOS uses.
- mrlonglong 4y ago
  Quite,you set up the interrupt vectors and mask off the video memory regions so access throws an exception and all this can be handled in the exception handlers