6 ms·

I studied MS-DOS inside and out (quite literally). One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes b

by ksaj 4y ago

I studied MS-DOS inside and out (quite literally). One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point, called the Program Segment Prefix (PSP). It is why you could simply execute a RET (return) and have it drop back to DOS again. The first bytes of the PSP are a literal INT 20h which is the Drop To DOS interrupt, and a RET with nothing left on the stack will return you to that zero offset, and then Drop to DOS. Probably a bit hacky, but really clever. I wrote my own DOS clone in Assembler, that I called PoverDOS because it was surely going to make me about that much money, and managed to make a boot sector that was far more functional than the one that comes standard with MS and PC DOS. The main thing it did was self-diagnostics to determine if the system had been infected by a boot sector virus. That was quite a feat, but became a moot point once 64 bit computing came about, because the boot sequence also gained a whole lot more space to work in. Don't bring it up with me at a party, 'cos all those DOS intricacies are still among my favourite subjects, and 97% of the population probably would not be nearly as enthusiastic. DOS is still the party killer it always was.

Dracophoenix 4y ago
How much of DOS did you learn by reading MS's raw source code vs reversing the OS yourself?
- ksaj 4y ago
  I glanced through it. I didn't learn anything new, except the definition of bloat. The raw source wasn't available when I was doing this, so I learned it entirely by reverse engineering, and documentation. Ironically I learned how the boot sector worked and basic ASM purely by using DEBUG.EXE. My computer got infected by Stoned virus, and I was pretty excited about it! This was back when DOS came with a green binder with actual documentation. It was a year or so of that before I discovered TASM and then MASM assemblers, which of course are a LOT more human-friendly. The earlier versions of DOS were entirely written in ASM, and they were super-tight. But later versions were all rewritten in high level languages. The stuff in the leak is all the bloated high-level language versions.
Aloha 4y ago
You and I would have good party conversations
- fuzzfactor 4y ago
  Well tonight we're going to party like it's 1999. This article was originally published right before the issuance of MS-DOS 2.0. That was not even near the '90's yet. By the time this was captured in the year 2000, the MSDOS.SYS file had already become almost a dummy file of few bytes in Win9x. The DOS OS itself with only the native commands is basically contained within the IO.SYS, MSDOS.SYS, and COMMAND.COM files, originally written in that order starting at the first working sector of a formatted floppy disk so it would boot. On a hard drive the 3 files would have to be written starting at the first working formatted sector of an actively flagged primary partition, and a proper master boot record needs to be on sector 0 (along with a valid partition table) if any boot files on that HDD are to be recognized by BIOS to begin with. Otherwise it would not boot. IOW once the volume boots you can do things like copy files and list directories, but if you want to edit files or format other volumes you needed more than just the 3 OS files. You needed the additional OS apps like EDIT.COM and FORMAT.COM on your floppy so they would be there when you needed them after you booted. Once booted, the SYS command could be used to properly write the OS to a target volume, so you needed SYS.COM for that. And by W9x, the three DOS OS files would no longer need to be right there at the start of the volume (as if they were the first things copied to the volume), and DOS FDISK would handle HDDs of hundreds of GB even though each FAT32 partition was only supported up to 32GB. The undocumented FDISK /MBR command executed from an A: or B: (floppy) prompt would simply write a fresh bootable DOS master boot record to sector 0 of C: (what should usually be considered) your only HDD on the mainboard. Without disturbing the partition table there. The SYS command also replaces the Volume Boot Record (bootsector) of the target volume with a fresh DOS bootsector as it freshly (over)writes IO.SYS, MSDOS.SYS, and COMMAND.COM into the file system. So you want to make a zeroed HDD boot, first boot to a DOS floppy (USB stick will do): Run FDISK and create a primary partition (bigger than 512MB) smaller than 32GB on the fixed (HDD) disk. Make the partition Active. (Alternatively you could use some other way to make a suitable partition, like Linux or a modern Windows version but the resulting "geometry" of the disk layout may not be the same or fully recognizable by DOS, which can affect the bootability. Sometimes with Linux or Windows, the older the version the better. Outcomes can also be surprisingly interesting when certain sectors of the HDD are not exactly zeroed beforehand. AMD also not quite the same as Intel, nor old Seagate identical to WD.) Now you've got a partition table on sector 0 with one primary partition marked "active" so the BIOS will boot to it. Run FDISK /MBR and it silently writes an MBR to sector 0 so the BIOS will REALLY boot to it. (You may need to manually copy a working MBR to the first 440 bytes of sector 0 if the FDISK /MBR command does not do the trick. The MBR is supposed to be generic for DOS and a Linux version like MBR.bin from the Syslinux project can be copied bytewise with binary tools. Newer Windows will also upgrade the MBR but it will still boot DOS (or Linux) afterward from a submenu.) All other sectors are still unchanged. Run FORMAT on the partition and I would always suggest labeling it as to HDD/Partition# when done. 11 characters max. Now you've got something other than zeros in places other than sector 0 of your HDD, but still no data actually located on the formatted volume yet. Run SYS on the formatted volume and it properly writes the 3 critical DOS OS files to the volume. Power down, remove the floppy or USB device. The HDD should then boot DOS according to mainboard BIOS priority. Add W9x to the same FAT32 volume and you still get a DOS boot entry on your Microsoft boot menu for when you want plain DOS. Install WXP to a different formatted partition (even FAT32) and your old W9x/DOS volume still can boot when desired as a submenu of the NT5 NTloader that replaces the DOS bootloader on that active partition. Then with Win10 (non-UEFI setup) on a third (NTFS) partition you can have an NTLDR submenu within the replacement NT6 Bootmgr and boot any of the above. Note the volume you installed the new Windows version to does not become active itself nor contain any boot files. By default additional boot entries are made to the established active partition. Even logical partitions containing OS's can be booted this way, according to OS version, from one of the few primary partitions you are allowed to flag as active. BIOS only. Too bad with foolish-type UEFI or GPT, no DOS for you. Also remember with bare-metal DOS/W9x and audio/MIDI (no USB), no latency for you either. Even on dog-slow PC's. Things like that.
ipython 4y ago
Awesome! I encourage you to post your code and blog about it. I would find it fascinating, and suspect others would too.
- ksaj 4y ago
  I didn't write a blog about it, but the code for the boot sector and installer (dropper) is here: https://github.com/ksaj/antivirus-mbr https://github.com/ksaj/antivirus-mbr I separated it out because it was useful for all versions of DOS at the time, and boot sectors, being outside the DOS realm, were always a sitting duck. I documented it to death, so it is super easy to read even though it is ASM code.
zozbot234 4y ago
> One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point ... AIUI that design was in CP/M already, and retained for compatibility.
- rzzzt 4y ago
  The two File Control Blocks are also vestigial remains of CP/M, and they can also be found in the same section of memory.
hutzlibu 4y ago
"and 97% of the population probably would not be nearly as enthusiastic" Thats likely a strong understatement.
mysterydip 4y ago
Did you ever write a book? I'd love to read up on this kind of stuff.
- hyperman1 4y ago
  Other people did e.g Peter Norton wrote a whole library around this stuff.
stevekemp 4y ago
For me my reversing started and ending with discovering and documenting the List of Lists: https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html That allowed TSRs to do clever things. I didn't use CP/M before DOS, but these days I know all about the PSP via writing code for CP/M, in Z80. Kinda impressive how long all these backwards compatibility things last.
- mickeyp 4y ago
  One of the first things I looked up, as a budding Turbo C developer in the DOS era, as it held all manner of secrets. I seem to recall 'one guy' had this hand-curated master list of interrupts. Might've been this list; perhaps another. I was obsessed with TSR programs back then. The notion that you can run a background process (in exchange for gobbling up all-valuable conventional memory) thanks to clever hacking really captured my eye back then.
  - mrlonglong 4y ago
    Chap you're referring to is Ralf Brown, he published a list of all known interrupts. It is on the web https://www.cs.cmu.edu/~ralf/files.html https://www.cs.cmu.edu/~ralf/files.html
    - ksaj 4y ago
      There was a pair of books that were instrumental to me: Undocumented PC and Undocumented DOS. Those books taught me so many tricks! I still remember the DWIM (Do what I mean) interrupt they used as the demonstration of how each interrupt was documented.
      - rasz 4y ago
        The Undocumented PC, A Programmer's Guide to I/O, CPUs, and Fixed Memory Areas by Frank van Gilluwe is great. Frank van Gilluwe is the author of Sourcer disassembler https://corexor.wordpress.com/2015/12/09/sourcer-and-windows-source/ https://corexor.wordpress.com/2015/12/09/sourcer-and-windows...

by ksaj 4y ago

Dracophoenix 4y ago

How much of DOS did you learn by reading MS's raw source code vs reversing the OS yourself?

ksaj 4y ago
I glanced through it. I didn't learn anything new, except the definition of bloat. The raw source wasn't available when I was doing this, so I learned it entirely by reverse engineering, and documentation. Ironically I learned how the boot sector worked and basic ASM purely by using DEBUG.EXE. My computer got infected by Stoned virus, and I was pretty excited about it! This was back when DOS came with a green binder with actual documentation. It was a year or so of that before I discovered TASM and then MASM assemblers, which of course are a LOT more human-friendly. The earlier versions of DOS were entirely written in ASM, and they were super-tight. But later versions were all rewritten in high level languages. The stuff in the leak is all the bloated high-level language versions.

Aloha 4y ago

You and I would have good party conversations

fuzzfactor 4y ago
Well tonight we're going to party like it's 1999. This article was originally published right before the issuance of MS-DOS 2.0. That was not even near the '90's yet. By the time this was captured in the year 2000, the MSDOS.SYS file had already become almost a dummy file of few bytes in Win9x. The DOS OS itself with only the native commands is basically contained within the IO.SYS, MSDOS.SYS, and COMMAND.COM files, originally written in that order starting at the first working sector of a formatted floppy disk so it would boot. On a hard drive the 3 files would have to be written starting at the first working formatted sector of an actively flagged primary partition, and a proper master boot record needs to be on sector 0 (along with a valid partition table) if any boot files on that HDD are to be recognized by BIOS to begin with. Otherwise it would not boot. IOW once the volume boots you can do things like copy files and list directories, but if you want to edit files or format other volumes you needed more than just the 3 OS files. You needed the additional OS apps like EDIT.COM and FORMAT.COM on your floppy so they would be there when you needed them after you booted. Once booted, the SYS command could be used to properly write the OS to a target volume, so you needed SYS.COM for that. And by W9x, the three DOS OS files would no longer need to be right there at the start of the volume (as if they were the first things copied to the volume), and DOS FDISK would handle HDDs of hundreds of GB even though each FAT32 partition was only supported up to 32GB. The undocumented FDISK /MBR command executed from an A: or B: (floppy) prompt would simply write a fresh bootable DOS master boot record to sector 0 of C: (what should usually be considered) your only HDD on the mainboard. Without disturbing the partition table there. The SYS command also replaces the Volume Boot Record (bootsector) of the target volume with a fresh DOS bootsector as it freshly (over)writes IO.SYS, MSDOS.SYS, and COMMAND.COM into the file system. So you want to make a zeroed HDD boot, first boot to a DOS floppy (USB stick will do): Run FDISK and create a primary partition (bigger than 512MB) smaller than 32GB on the fixed (HDD) disk. Make the partition Active. (Alternatively you could use some other way to make a suitable partition, like Linux or a modern Windows version but the resulting "geometry" of the disk layout may not be the same or fully recognizable by DOS, which can affect the bootability. Sometimes with Linux or Windows, the older the version the better. Outcomes can also be surprisingly interesting when certain sectors of the HDD are not exactly zeroed beforehand. AMD also not quite the same as Intel, nor old Seagate identical to WD.) Now you've got a partition table on sector 0 with one primary partition marked "active" so the BIOS will boot to it. Run FDISK /MBR and it silently writes an MBR to sector 0 so the BIOS will REALLY boot to it. (You may need to manually copy a working MBR to the first 440 bytes of sector 0 if the FDISK /MBR command does not do the trick. The MBR is supposed to be generic for DOS and a Linux version like MBR.bin from the Syslinux project can be copied bytewise with binary tools. Newer Windows will also upgrade the MBR but it will still boot DOS (or Linux) afterward from a submenu.) All other sectors are still unchanged. Run FORMAT on the partition and I would always suggest labeling it as to HDD/Partition# when done. 11 characters max. Now you've got something other than zeros in places other than sector 0 of your HDD, but still no data actually located on the formatted volume yet. Run SYS on the formatted volume and it properly writes the 3 critical DOS OS files to the volume. Power down, remove the floppy or USB device. The HDD should then boot DOS according to mainboard BIOS priority. Add W9x to the same FAT32 volume and you still get a DOS boot entry on your Microsoft boot menu for when you want plain DOS. Install WXP to a different formatted partition (even FAT32) and your old W9x/DOS volume still can boot when desired as a submenu of the NT5 NTloader that replaces the DOS bootloader on that active partition. Then with Win10 (non-UEFI setup) on a third (NTFS) partition you can have an NTLDR submenu within the replacement NT6 Bootmgr and boot any of the above. Note the volume you installed the new Windows version to does not become active itself nor contain any boot files. By default additional boot entries are made to the established active partition. Even logical partitions containing OS's can be booted this way, according to OS version, from one of the few primary partitions you are allowed to flag as active. BIOS only. Too bad with foolish-type UEFI or GPT, no DOS for you. Also remember with bare-metal DOS/W9x and audio/MIDI (no USB), no latency for you either. Even on dog-slow PC's. Things like that.

ipython 4y ago

Awesome! I encourage you to post your code and blog about it. I would find it fascinating, and suspect others would too.

ksaj 4y ago
I didn't write a blog about it, but the code for the boot sector and installer (dropper) is here: https://github.com/ksaj/antivirus-mbr https://github.com/ksaj/antivirus-mbr I separated it out because it was useful for all versions of DOS at the time, and boot sectors, being outside the DOS realm, were always a sitting duck. I documented it to death, so it is super easy to read even though it is ASM code.

zozbot234 4y ago

> One thing that I always thought was interesting was all the stuff that gets packed in the first 100h bytes before a .COM file's entry point ... AIUI that design was in CP/M already, and retained for compatibility.

rzzzt 4y ago
The two File Control Blocks are also vestigial remains of CP/M, and they can also be found in the same section of memory.

hutzlibu 4y ago

"and 97% of the population probably would not be nearly as enthusiastic" Thats likely a strong understatement.

mysterydip 4y ago

Did you ever write a book? I'd love to read up on this kind of stuff.

hyperman1 4y ago
Other people did e.g Peter Norton wrote a whole library around this stuff.

stevekemp 4y ago

For me my reversing started and ending with discovering and documenting the List of Lists: https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html https://fd.lod.bz/rbil/interrup/dos_kernel/2152.html That allowed TSRs to do clever things. I didn't use CP/M before DOS, but these days I know all about the PSP via writing code for CP/M, in Z80. Kinda impressive how long all these backwards compatibility things last.

mickeyp 4y ago
One of the first things I looked up, as a budding Turbo C developer in the DOS era, as it held all manner of secrets. I seem to recall 'one guy' had this hand-curated master list of interrupts. Might've been this list; perhaps another. I was obsessed with TSR programs back then. The notion that you can run a background process (in exchange for gobbling up all-valuable conventional memory) thanks to clever hacking really captured my eye back then.
- mrlonglong 4y ago
  Chap you're referring to is Ralf Brown, he published a list of all known interrupts. It is on the web https://www.cs.cmu.edu/~ralf/files.html https://www.cs.cmu.edu/~ralf/files.html
  - ksaj 4y ago
    There was a pair of books that were instrumental to me: Undocumented PC and Undocumented DOS. Those books taught me so many tricks! I still remember the DWIM (Do what I mean) interrupt they used as the demonstration of how each interrupt was documented.
    - rasz 4y ago
      The Undocumented PC, A Programmer's Guide to I/O, CPUs, and Fixed Memory Areas by Frank van Gilluwe is great. Frank van Gilluwe is the author of Sourcer disassembler https://corexor.wordpress.com/2015/12/09/sourcer-and-windows-source/ https://corexor.wordpress.com/2015/12/09/sourcer-and-windows...