7 ms·

It feels weird that so much old software from the 80's or even 90's and newer has to be reverse-engineered in order to be understood. The process almost seems l

It feels weird that so much old software from the 80's or even 90's and newer has to be reverse-engineered in order to be understood. The process almost seems like something an archeologist would perform on an artifact thousands of years old. But this software likely has people living who actually built the stuff and know how it works. I understand not everyone can be bothered in life to commit to a project, but surely a phone call could be answered occasionally. Is it just that knowing who these people are is too difficult or that existing copyright issues simply make it unsafe?

Gordonjcp 4y ago
In my quest to reverse-engineer the Ensoniq Mirage software I reached out to one of the original authors, who replied to my email and said that while he remembered working on the Mirage fondly he couldn't remember any details and no longer had any of his notes or paperwork from the 1980s.
- KerrAvon 4y ago
  In fairness, I've found this also tends to be the answer one receives when asking the author of some code from two years ago about some important details. Comment your code well, kids. Remember to document intent -- the why, not the what.
  - rbanffy 4y ago
    "Comment your code as if a violent sociopath who knows where you live were going to maintain it". Eventually one will.
- joezydeco 4y ago
  It's a sensible reply when you don't want to be accused of holding onto IP that you don't own. I have kept source code from many projects that I've worked on but if some stranger asked me for details about any of them I'd give the exact same response. It could also be the complete truth.
  - Gordonjcp 4y ago
    At this point the company that bought the company that bought Ensoniq is gone, so it's hard to imagine who would come looking for 12kB worth of 6809 assembly code.
    - rbanffy 4y ago
      Any patents about it should also have expired by now.
- lproven 4y ago
  I can well believe that. I moderated a talk between 4 of the original team members at Acorn that wrote RISC OS [explanation below] in the 1980s a few months ago. They're all of retirement age now. Three of them were full of anecdotes and stories and details about the company, the machines, the buildings, etc. The other could barely remember anything about it; his former colleagues were prompting him for details of what he wrote about 35Y ago, but he had little idea. Too much other work since, he said, and too long ago, and he'd forgotten. I hang out in various retrocomputing fora for fun, and it amazes me the utter nonsense that some people come up with from vaguely-remembered things they knew 30Y ago. They don't know that they've forgotten, so they free-associate stuff and just make it up, and then many get resentful when their wildly wrong answers are corrected. RISC OS, for youngsters and Americans (where Acorn didn't sell much) is the original native OS for the ARM CPU, written by the company that developed the CPU and the machine it would power. It is still around today, it's FOSS now, and it runs on some modern ARM hardware such as the Raspberry Pi. It is a multitasking GUI OS with a rich desktop, networking, internet support, and so on. It ran in from 512kB to 4MB of RAM. It was the first ever GUI computer to anti-alias screen fonts as standard, to move the whole window when it was dragged, rather than a dotted outline and then a redraw when the mouse button was released. The OS, its GUI, its core apps (text editor, image editor, console prompt, filer, etc.) fitted into and executed directly from ROM, that is, without being copied into RAM. It was also the first GUI with a dedicated panel showing icons for running apps, plus a system control menu, a clock, and so on -- it inspired the Dock in NeXTstep (which came out 2Y later), which both inspired the taskbar in Windows 95 (which came out another 7Y after that). And the whole OS was written by about 6-7 programmers, and the vast majority of it in hand-coded assembly language.
  - selestify 4y ago
    What retro computing forums are you on?
    - lproven 4y ago
      Me? The ClassicCmp mailing lists, among others. Half a dozen subReddits and a dozen Facebook groups. A handful of Usenet groups. I don't use web fora much so I won't count them.
pvg 4y ago
People tend not to remember the details of things they worked on decades ago, let alone have access to source code or contemporary documentation. Even Jordan Mechner - enough of a design-note-taker and packrat to produce detailed retellings of his development process many years later misplaced the original source to Karateka.
KerrAvon 4y ago
Assuming that someone has saved the source, you'd have to track them down. If it was work for hire, like say at a fruit company, that person probably doesn't have the rights to it. So you have to go to the lawyers. The fruit company lawyers might not actually be able to verify that the fruit company owns it, but if they want to try, it will require some effort to find the documentation, which will be somewhere in dead tree-based archives from the 1980's if it exists at all. Which is to say that if you can identify the rights holder and it's not an individual, you have to convince them to spend time and money to release the rights. That they may not be able to easily confirm they have. Or you could grab the binary and disassemble it and figure out what it's doing yourself.
- Brian_K_White 4y ago
  I think we all know all of that. I think the point of the remark is specifically to step back from those obvious givens and observe an essential absurdity.
  - SllX 4y ago
    It’s not that absurd that people conserve their actions, time and money and a different group of people spend their own time and money on understanding something that they value differently than the people who originated the software. You could impose obligations on people to release more than they do, but that will just change the calculus on what they choose to work on and release at all because now other people want to spend their time for them. This is just an inconvenience.
    - Brian_K_White 4y ago
      What? This is some weird overblown reaction. The absurdity is only a small one and does not imply any kind of suggestion about imposing obligations or anything like that. Again, the details of how the situation came to be are not any mystery. The end result, forget the chain of reasonable steps to get there, is in a sense silly, or perhaps remarkable, in the literal sense of simply "bears remarking", that one person is studying something like an archeologist assembling guesses and deductions from incomplete clues, while another person exists somewhere in the world who is physically alive and posesses the the answers natively that the other is guessing at. No one said that it was a crime that demands action, and no one said anyone's individual actions, either the original developers or the "archeologists", are absurd.
TMWNN 4y ago
>It feels weird that so much old software from the 80's or even 90's and newer has to be reverse-engineered in order to be understood. The process almost seems like something an archeologist would perform on an artifact thousands of years old. This is a plot point in Vernor Vinge's A Deepness in the Sky. Thousands of years in the future, pretty much all software has already been written, so software archaeologists trawl through archives to find, debug, and run code (in as many emulated environments as needed).
- retrac 4y ago
  Wonderful novel. Galactic standard time is still counted in the number of seconds since a certain epoch. (January 1st 1970 00:00 UTC as it happens, though both that calendar system, and the precise significance, if any, of the date, are lost to history by the time of the novel. The first moon landing, maybe? So few records from that time survive.)
  - thrtythreeforty 4y ago
    Qeng Ho tradition holds that the epoch is the first moon landing, but Pham looked closer (he is a programmer-at-arms, it's his job to know these things) and saw that 1970 was the epoch.
scarface74 4y ago
Someone please ELI5. I was no savant at 13 years old. But I remember being able to trace through 65C02 code and figure out enough of it to hack around and fix Software Automated Mouth to work in 80 column mode. When you redirected output from SAM to the screen it would revert to 40 column mode. Later on, I ported it to work with ProDOS. https://en.wikipedia.org/wiki/Software_Automatic_Mouth https://en.wikipedia.org/wiki/Software_Automatic_Mouth I remember also hacking around with other 65C02 assembly language applications and making modifications. Over the next 10 years, I worked with other processors in assembly (68K, PPC, x86) and 65C02 was simple. Especially since most programs were written in 65C02 and not compiled down to it.
- Someone 4y ago
  68k isn’t hard to understand, either. I think what helped most was that programs were small and that no tricks were necessary or even possible to be efficient w.r.t. cache misses. If Linux ran on some multi-gigahertz 64-bit evolution of the 6502 the instruction set would still be simple, but a few Gigabytes of 6502 code still would be a challenge to understand, more so if that 6502 were out of order, speculatively executing, multi-core, and various parts of the code were written taking that into account.
  - zozbot234 4y ago
    Indeed, the 68k ISA is comparatively clean for a traditional CISC architecture, but early RISC were regarded as even simpler and easier to understand.
wikitopian 4y ago
Shit, I can't even tell you what I was trying to do with software I wrote a week ago. I imagine the person who wrote it knows about as much about the code as the person calling them.
userbinator 4y ago
What are you going to believe, the instructions in the binary itself or what someone may or may not remember when it was written? The latter is good if you want to have some historical context around certain decisions, but the former is far more concrete when trying to understand how it works and how to modify it.
pabs3 4y ago
Software Heritage are solving this problem for current software by archiving software source code. I think that at one point they were talking about doing source code escrow, for folks who write proprietary software. https://www.softwareheritage.org/ https://www.softwareheritage.org/
jhbadger 4y ago
One thing to remember is that most Apple ][ software barely had source code as we would understand it today, as most code was written in 6502 assembly language rather than in a high level language (compilers at the time took too much memory and generated poor code anyway). So even if you had access to the original source pretty much the only advantage over using a disassembly would be that there might be a handful of comments.
- kazinator 4y ago
  In assembly you also have meaningful labels for branch destinations and routines in the program and global variables. Named constants for offsets into tables, bitmasks, and such. The original assembly language source could greatly ease reverse engineering, especially if it uses good naming. Some assemblers have macros; it's much better to be able to read the macro invocations than their assembled expansions.

Gordonjcp 4y ago

In my quest to reverse-engineer the Ensoniq Mirage software I reached out to one of the original authors, who replied to my email and said that while he remembered working on the Mirage fondly he couldn't remember any details and no longer had any of his notes or paperwork from the 1980s.

KerrAvon 4y ago
In fairness, I've found this also tends to be the answer one receives when asking the author of some code from two years ago about some important details. Comment your code well, kids. Remember to document intent -- the why, not the what.
- rbanffy 4y ago
  "Comment your code as if a violent sociopath who knows where you live were going to maintain it". Eventually one will.
joezydeco 4y ago
It's a sensible reply when you don't want to be accused of holding onto IP that you don't own. I have kept source code from many projects that I've worked on but if some stranger asked me for details about any of them I'd give the exact same response. It could also be the complete truth.
- Gordonjcp 4y ago
  At this point the company that bought the company that bought Ensoniq is gone, so it's hard to imagine who would come looking for 12kB worth of 6809 assembly code.
  - rbanffy 4y ago
    Any patents about it should also have expired by now.
lproven 4y ago
I can well believe that. I moderated a talk between 4 of the original team members at Acorn that wrote RISC OS [explanation below] in the 1980s a few months ago. They're all of retirement age now. Three of them were full of anecdotes and stories and details about the company, the machines, the buildings, etc. The other could barely remember anything about it; his former colleagues were prompting him for details of what he wrote about 35Y ago, but he had little idea. Too much other work since, he said, and too long ago, and he'd forgotten. I hang out in various retrocomputing fora for fun, and it amazes me the utter nonsense that some people come up with from vaguely-remembered things they knew 30Y ago. They don't know that they've forgotten, so they free-associate stuff and just make it up, and then many get resentful when their wildly wrong answers are corrected. RISC OS, for youngsters and Americans (where Acorn didn't sell much) is the original native OS for the ARM CPU, written by the company that developed the CPU and the machine it would power. It is still around today, it's FOSS now, and it runs on some modern ARM hardware such as the Raspberry Pi. It is a multitasking GUI OS with a rich desktop, networking, internet support, and so on. It ran in from 512kB to 4MB of RAM. It was the first ever GUI computer to anti-alias screen fonts as standard, to move the whole window when it was dragged, rather than a dotted outline and then a redraw when the mouse button was released. The OS, its GUI, its core apps (text editor, image editor, console prompt, filer, etc.) fitted into and executed directly from ROM, that is, without being copied into RAM. It was also the first GUI with a dedicated panel showing icons for running apps, plus a system control menu, a clock, and so on -- it inspired the Dock in NeXTstep (which came out 2Y later), which both inspired the taskbar in Windows 95 (which came out another 7Y after that). And the whole OS was written by about 6-7 programmers, and the vast majority of it in hand-coded assembly language.
- selestify 4y ago
  What retro computing forums are you on?
  - lproven 4y ago
    Me? The ClassicCmp mailing lists, among others. Half a dozen subReddits and a dozen Facebook groups. A handful of Usenet groups. I don't use web fora much so I won't count them.

pvg 4y ago

People tend not to remember the details of things they worked on decades ago, let alone have access to source code or contemporary documentation. Even Jordan Mechner - enough of a design-note-taker and packrat to produce detailed retellings of his development process many years later misplaced the original source to Karateka.

KerrAvon 4y ago

Assuming that someone has saved the source, you'd have to track them down. If it was work for hire, like say at a fruit company, that person probably doesn't have the rights to it. So you have to go to the lawyers. The fruit company lawyers might not actually be able to verify that the fruit company owns it, but if they want to try, it will require some effort to find the documentation, which will be somewhere in dead tree-based archives from the 1980's if it exists at all. Which is to say that if you can identify the rights holder and it's not an individual, you have to convince them to spend time and money to release the rights. That they may not be able to easily confirm they have. Or you could grab the binary and disassemble it and figure out what it's doing yourself.

Brian_K_White 4y ago
I think we all know all of that. I think the point of the remark is specifically to step back from those obvious givens and observe an essential absurdity.
- SllX 4y ago
  It’s not that absurd that people conserve their actions, time and money and a different group of people spend their own time and money on understanding something that they value differently than the people who originated the software. You could impose obligations on people to release more than they do, but that will just change the calculus on what they choose to work on and release at all because now other people want to spend their time for them. This is just an inconvenience.
  - Brian_K_White 4y ago
    What? This is some weird overblown reaction. The absurdity is only a small one and does not imply any kind of suggestion about imposing obligations or anything like that. Again, the details of how the situation came to be are not any mystery. The end result, forget the chain of reasonable steps to get there, is in a sense silly, or perhaps remarkable, in the literal sense of simply "bears remarking", that one person is studying something like an archeologist assembling guesses and deductions from incomplete clues, while another person exists somewhere in the world who is physically alive and posesses the the answers natively that the other is guessing at. No one said that it was a crime that demands action, and no one said anyone's individual actions, either the original developers or the "archeologists", are absurd.

TMWNN 4y ago

>It feels weird that so much old software from the 80's or even 90's and newer has to be reverse-engineered in order to be understood. The process almost seems like something an archeologist would perform on an artifact thousands of years old. This is a plot point in Vernor Vinge's A Deepness in the Sky. Thousands of years in the future, pretty much all software has already been written, so software archaeologists trawl through archives to find, debug, and run code (in as many emulated environments as needed).

retrac 4y ago
Wonderful novel. Galactic standard time is still counted in the number of seconds since a certain epoch. (January 1st 1970 00:00 UTC as it happens, though both that calendar system, and the precise significance, if any, of the date, are lost to history by the time of the novel. The first moon landing, maybe? So few records from that time survive.)
- thrtythreeforty 4y ago
  Qeng Ho tradition holds that the epoch is the first moon landing, but Pham looked closer (he is a programmer-at-arms, it's his job to know these things) and saw that 1970 was the epoch.

scarface74 4y ago

Someone please ELI5. I was no savant at 13 years old. But I remember being able to trace through 65C02 code and figure out enough of it to hack around and fix Software Automated Mouth to work in 80 column mode. When you redirected output from SAM to the screen it would revert to 40 column mode. Later on, I ported it to work with ProDOS. https://en.wikipedia.org/wiki/Software_Automatic_Mouth https://en.wikipedia.org/wiki/Software_Automatic_Mouth I remember also hacking around with other 65C02 assembly language applications and making modifications. Over the next 10 years, I worked with other processors in assembly (68K, PPC, x86) and 65C02 was simple. Especially since most programs were written in 65C02 and not compiled down to it.

Someone 4y ago
68k isn’t hard to understand, either. I think what helped most was that programs were small and that no tricks were necessary or even possible to be efficient w.r.t. cache misses. If Linux ran on some multi-gigahertz 64-bit evolution of the 6502 the instruction set would still be simple, but a few Gigabytes of 6502 code still would be a challenge to understand, more so if that 6502 were out of order, speculatively executing, multi-core, and various parts of the code were written taking that into account.
- zozbot234 4y ago
  Indeed, the 68k ISA is comparatively clean for a traditional CISC architecture, but early RISC were regarded as even simpler and easier to understand.

wikitopian 4y ago

Shit, I can't even tell you what I was trying to do with software I wrote a week ago. I imagine the person who wrote it knows about as much about the code as the person calling them.

userbinator 4y ago

What are you going to believe, the instructions in the binary itself or what someone may or may not remember when it was written? The latter is good if you want to have some historical context around certain decisions, but the former is far more concrete when trying to understand how it works and how to modify it.

pabs3 4y ago

Software Heritage are solving this problem for current software by archiving software source code. I think that at one point they were talking about doing source code escrow, for folks who write proprietary software. https://www.softwareheritage.org/ https://www.softwareheritage.org/

jhbadger 4y ago

One thing to remember is that most Apple ][ software barely had source code as we would understand it today, as most code was written in 6502 assembly language rather than in a high level language (compilers at the time took too much memory and generated poor code anyway). So even if you had access to the original source pretty much the only advantage over using a disassembly would be that there might be a handful of comments.

kazinator 4y ago
In assembly you also have meaningful labels for branch destinations and routines in the program and global variables. Named constants for offsets into tables, bitmasks, and such. The original assembly language source could greatly ease reverse engineering, especially if it uses good naming. Some assemblers have macros; it's much better to be able to read the macro invocations than their assembled expansions.