3 ms·

GCR Decoding on the Fly

27 points by mmastrac 4y ago

ajross 4y ago
This is great, but requires a ton of background to understand what the author is doing and why. The Commodore 64 had a... somewhat unique floppy disk architecture. The disk drive itself had its own 6502 processor and some RAM, and so Commodore felt they didn't need to buy a WDC or NEC floppy controller, they could do all the work of interpreting the 250 kHz bit stream (which for physics reasons can't be the raw bit pattern the user wants to store) from the device in software. Woz had pioneered this technique four years earlier, so they knew it could work even though they made a few different and somewhat questionable choices in the hardware design. That's the "GCR" encoding being discussed. Also, having a separate CPU meant that all the filesystem[1] duties were actually farmed off to the drive, the main CPU would just speak what amounts to a system call (open/close/read/write) interface over a 1 bit synchronous interface called IEC. But that interface was slow. Hardware bugs with their interface chip meant it needed to be bit-banged from the 1 MHz 6502's. Effectively it was a 1 bit wide parallel port, and it was awful (like, literally 10x slower at disk I/O than an Apple II!). BUT: the separate CPU had commands that would load and execute code from the host, so its behavior could be arbitrarily changed. So... a whole ecosystem of "fast loader" products grew up using deep mysteries to work around this. Most of them were basically two-bit clockless/async transfers written in extremely carefully timed assembly. But because of that crazy-accurate sub-cycle timing requirement, you couldn't be doing stuff like "reading the disk" and "decoding the flux patterns" at the same time. Fast loaders would load the raw data out of the shift register and only when the full sector was in RAM do the work of unpacking it into bytes to be sent over the IEC wires. So even though the transfer was fast there was a big latency cost. So that's what the author is doing: reading bytes off the disk and decoding them simultaneously. It's a pretty amazing trick. [1] That's the word we use today. Commodore just called it "DOS", which was ambiguous even then.
- a1369209993 4y ago
  > the 250 kHz bit stream (which for physics reasons can't be the raw bit pattern the user wants to store) Technically it's more reliability/engineering reasons. Physics doesn't care (you can basically write whatever bit pattern you like), but if there are more than two consecutive zeroes in the raw bitstream, the timing hardware can potentially be unsure if it was supposed to be three zeroes or four.
  - ajross 4y ago
    That's true, but there's also a minimum limit. Put the flux transitions too close to each other and they'll disappear (the iron in the middle will reorient). Also it's not just clock timing that is a problem, the differential amplifier circuit used is a feedback thing. If it doesn't get input for long enough, it will start amplifying noise in the environment (devices with motors have a LOT of noise) and generate false pulses with essentially random timings. This was the source of "weak sectors" used in copy protection: you'd use special hardware to essentially erase a region in the middle of the sector, and the drive would report a bunch of 1's anyway, but never the same bits twice. Consumer drives couldn't reproduce that behavior, they only knew how to write "correct" bit patterns with clock bits. In practice, the media of the time could handle flux transitions as close as 4us (on a 300RPM drive) or as far apart as ~12 (though only Apple was that brave, because they had a self-clocking circuit, other vendors stuck to 8us. MFM encodings would soon push that back up to 12, and eventually the "HD" encodings would double the clock rate to 2-6us (but they didn't use the extra time otherwise, RLL was only ever deployed on fixed media).
    - a1369209993 4y ago
      > Put the flux transitions too close to each other and they'll disappear Sure, but that's below the bit pattern. For the bit stream, flux transitions can never be closer together than two consecutive one bits, which is fine and happens all the time (at least for the media GCR applies to; there are other encoding that have a minimum number of zeroes between ones in addition to a maximum). > the differential amplifier circuit used is a feedback thing. If it doesn't get input for long enough, it will start amplifying noise in the environment Fair point, although I'd still call that a engineering issue rather than a physics one - you can write a "weak sector" to disk, and in principle read it back, it's just that the hardware in 1541 isn't up to the task, for legitimate engineering reasons but not beacause it's physically impossible.
      - ajross 4y ago
        > For the bit stream, flux transitions can never be closer together than two consecutive one bits, For the GCR encoding here, yes (also FM and Apple's GCR). But MFM ("double density") was contemporary, in use on the same media, based on a doubled sampling rate and a minimum of two sampling clocks between flux transitions. A bunch of logic ones there is encoded "101010101010..." (and zeros would be "0101010101...", etc...). Fancier RLL hard drive controllers woulds soon push that to a minimum of three, too. (And surely higher, but by that point the flux encoding had become a trade secret behind interfaces like SCSI or ATA, so it was never documented). "It's complicated", basically
dmitrygr 4y ago
Using glitch opcodes of the processor is a very nice touch! For those who are not aware: 6502 was hand-designed. There are many opcodes that are not documented (they were the "do not care" squares in the karnaugh maps and PALs used to create various internal control signals). This means that many still do something. Over the years many were documented well, and this code uses some of them, which is super cool! It does mean that this code will not work on later 6502 variants like the 65c02 which act differently when faced with those opcode.

27 points by mmastrac 4y ago

This is great, but requires a ton of background to understand what the author is doing and why. The Commodore 64 had a... somewhat unique floppy disk architecture. The disk drive itself had its own 6502 processor and some RAM, and so Commodore felt they didn't need to buy a WDC or NEC floppy controller, they could do all the work of interpreting the 250 kHz bit stream (which for physics reasons can't be the raw bit pattern the user wants to store) from the device in software. Woz had pioneered this technique four years earlier, so they knew it could work even though they made a few different and somewhat questionable choices in the hardware design. That's the "GCR" encoding being discussed. Also, having a separate CPU meant that all the filesystem[1] duties were actually farmed off to the drive, the main CPU would just speak what amounts to a system call (open/close/read/write) interface over a 1 bit synchronous interface called IEC. But that interface was slow. Hardware bugs with their interface chip meant it needed to be bit-banged from the 1 MHz 6502's. Effectively it was a 1 bit wide parallel port, and it was awful (like, literally 10x slower at disk I/O than an Apple II!). BUT: the separate CPU had commands that would load and execute code from the host, so its behavior could be arbitrarily changed. So... a whole ecosystem of "fast loader" products grew up using deep mysteries to work around this. Most of them were basically two-bit clockless/async transfers written in extremely carefully timed assembly. But because of that crazy-accurate sub-cycle timing requirement, you couldn't be doing stuff like "reading the disk" and "decoding the flux patterns" at the same time. Fast loaders would load the raw data out of the shift register and only when the full sector was in RAM do the work of unpacking it into bytes to be sent over the IEC wires. So even though the transfer was fast there was a big latency cost. So that's what the author is doing: reading bytes off the disk and decoding them simultaneously. It's a pretty amazing trick. [1] That's the word we use today. Commodore just called it "DOS", which was ambiguous even then.

a1369209993 4y ago
> the 250 kHz bit stream (which for physics reasons can't be the raw bit pattern the user wants to store) Technically it's more reliability/engineering reasons. Physics doesn't care (you can basically write whatever bit pattern you like), but if there are more than two consecutive zeroes in the raw bitstream, the timing hardware can potentially be unsure if it was supposed to be three zeroes or four.
- ajross 4y ago
  That's true, but there's also a minimum limit. Put the flux transitions too close to each other and they'll disappear (the iron in the middle will reorient). Also it's not just clock timing that is a problem, the differential amplifier circuit used is a feedback thing. If it doesn't get input for long enough, it will start amplifying noise in the environment (devices with motors have a LOT of noise) and generate false pulses with essentially random timings. This was the source of "weak sectors" used in copy protection: you'd use special hardware to essentially erase a region in the middle of the sector, and the drive would report a bunch of 1's anyway, but never the same bits twice. Consumer drives couldn't reproduce that behavior, they only knew how to write "correct" bit patterns with clock bits. In practice, the media of the time could handle flux transitions as close as 4us (on a 300RPM drive) or as far apart as ~12 (though only Apple was that brave, because they had a self-clocking circuit, other vendors stuck to 8us. MFM encodings would soon push that back up to 12, and eventually the "HD" encodings would double the clock rate to 2-6us (but they didn't use the extra time otherwise, RLL was only ever deployed on fixed media).
  - a1369209993 4y ago
    > Put the flux transitions too close to each other and they'll disappear Sure, but that's below the bit pattern. For the bit stream, flux transitions can never be closer together than two consecutive one bits, which is fine and happens all the time (at least for the media GCR applies to; there are other encoding that have a minimum number of zeroes between ones in addition to a maximum). > the differential amplifier circuit used is a feedback thing. If it doesn't get input for long enough, it will start amplifying noise in the environment Fair point, although I'd still call that a engineering issue rather than a physics one - you can write a "weak sector" to disk, and in principle read it back, it's just that the hardware in 1541 isn't up to the task, for legitimate engineering reasons but not beacause it's physically impossible.
    - ajross 4y ago
      > For the bit stream, flux transitions can never be closer together than two consecutive one bits, For the GCR encoding here, yes (also FM and Apple's GCR). But MFM ("double density") was contemporary, in use on the same media, based on a doubled sampling rate and a minimum of two sampling clocks between flux transitions. A bunch of logic ones there is encoded "101010101010..." (and zeros would be "0101010101...", etc...). Fancier RLL hard drive controllers woulds soon push that to a minimum of three, too. (And surely higher, but by that point the flux encoding had become a trade secret behind interfaces like SCSI or ATA, so it was never documented). "It's complicated", basically

dmitrygr 4y ago

Using glitch opcodes of the processor is a very nice touch! For those who are not aware: 6502 was hand-designed. There are many opcodes that are not documented (they were the "do not care" squares in the karnaugh maps and PALs used to create various internal control signals). This means that many still do something. Over the years many were documented well, and this code uses some of them, which is super cool! It does mean that this code will not work on later 6502 variants like the 65c02 which act differently when faced with those opcode.