4 ms·

I hope it's ok to answer here instead of on vcfed.org since I don't want to create yet another account... Sun workstations did not use the dual 68000 approach

I hope it's ok to answer here instead of on vcfed.org since I don't want to create yet another account... Sun workstations did not use the dual 68000 approach to handle VM. The problem with the 68000 was that it didn't save enough information on the stack when an exception occurred (e.g. a page fault), so a faulting instruction could not be recovered/re-executed in all cases. Sun solved this by using "harmless" instructions such as "tst" that did non have to be recovered in case of a page fault. The OS could then kick in, load the contents to memory and just skip the failed instruction. This required changes to the compiler, e.g. when creating a new stack frame that would have required to extend the stack size. For paging, this approach would cause significant overhead. However, the first versions of SunOS which ran on the Sun 1 workstations were based on 7th Edition Unix and used only swapping. The 68010, used in the Sun-2 workstation, fixed the restarting problem by saving sufficient state to the stack, so SunOS could switch to an early BSD release as its basis. AFAIK, the dual 68000 approach was used by early Apollo workstations instead. The way I understand it is that the second processor was idling until a page fault occurred. The 68000 has an asynchronous bus which requires an explicit acknowledge (/DTACK) to indicate that data (e.g. from slow memory) has arrived. This feature was used by the Apollos - the second 68k would kick in while /DTACK on the first 68k was deasserted, handle the required virtual memory operations, and then indicated that the first CPU can continue working. So here, no bus error was indicated (which would have caused the restarting problem), but the virtual memory behaved like a very slow physical memory. Apollos were able to perform remote memory accesses to other machines over the TokenRing interface (which was about as fast as memory accesses - 12 Mbit/s IIRC), so this feature was much more useful for DomainOS than for early Unix systems.

johndoe0815 3y ago
Please note that I can't confirm that Apollo used this approach - the earliest Apollo workstation I know (the DN300) used a 68010 already. I would love to hear more details on this... I'm also not sure which approach early SGI workstations used (the Iris 1000 series), documentation on those is extremely hard to find.
- Tor3 3y ago
  There's a Motorola document about this: https://ia904706.us.archive.org/5/items/Motorola-SeminarsandApplicationBooksDC-001VirtualMemoryUsingTheMC68000AndTheMC68451MMU/Motorola-SeminarsandApplicationBooksDC-001VirtualMemoryUsingTheMC68000AndTheMC68451MMU_text.pdf https://ia904706.us.archive.org/5/items/Motorola-Seminarsand... In there they suggest that "another bus master must perform this action" and that this one could be "an MC68000 MPU as well". I've also heard and seen the claim that Apollo workstations before the 68010 used this method, but it's difficult to find a definite source for this (though it's likely true. Every reference etc. to the pre-DN300 models claim 2x68000 CPU)
  - eschneider 3y ago
    I can confirm that the early Apollos did this. Worked on one back in the day.
- ThomasBHickey 3y ago
  They did indeed use multiple 68K chips. In fact I have one of the motherboards down in my basement (sadly not operational).
jonstewart 3y ago
This is great info. Can you elaborate or point to a link on the distinction between paging and swapping? I’ve always treated them as synonyms but clearly there are some differences on earlier systems.
- johndoe0815 3y ago
  Swapping (in the context of old Unix) is always replacing (copying) an entire process address space (or at least the non-shareable read-write segments such as .data), out to the swap device, or back, in one go. Paging is more advanced as it allows to keep only a part of the memory required by a process in RAM and contents can be loaded or moved to secondary storage on the granularity of single pages (typically 4 kB, but larger pages are also common nowadays). So a typical exec(2) system call in Unix could set up a new page table which contains no single byte of the executable in the beginning and then jump to the executable's entry point. This, in turn, results in a page fault, in turn, the OS can then load the requested page into RAM. Paging is more efficient since control/data flow analyses show that something similar to the Pareto rule (or 80/20 rule) also applies to programs which are executed - 80% of the time, only 20% of the address range (for instructions and/or data) is used. With paging, the OS won't need to load any code or data from disk that is never used (and can load it on demand later). What's probably confusing is that the area reserved on disk for storing RAM contents (a partition or a file) is still called "swap partition/file". This is an old term that refuses to die, I guess...
  - jonstewart 3y ago
    So “old” swapping is taking low priority processes and putting all of their memory out to disk, almost (but obviously not quite) as though you’re suspending them. Got it!
    - deleted 3y ago
      [deleted]
johndoe0815 3y ago
Here's an interesting discussion thread in comp.sys.apollo from 2003: https://groups.google.com/g/comp.sys.apollo/c/55dqpf11hEw/m/yvrhy2AlZVwJ https://groups.google.com/g/comp.sys.apollo/c/55dqpf11hEw/m/... The setup was more complex than I described, as the second 68000 did not simply wait for a page fault to happen but was executing regular code (unclear if application only or also OS kernel, but the code was not allowed to raise page faults, user processes usually ran on one CPU, see the last quote below) and only handled the fault on the other CPU when interrupted by the MMU. Unfortunately, the only versions of Aegis/DomainOS available online seem to be 10.x (on bitsavers), but support for the "sau1" 68000-based systems (DN100/DN400/DN416) ended with version 9.x, so we can't have a look at the code implementing this feature. Interesting quotes from the thread: "The two processors were called "A" and "B" and processes could be marked runnable on A or on B or both. It was almost a multiprocessor, except that you couldn't run anything that might take a page fault on both at the same time." (by Jim Rees, who runs the Apollo archive at https://jim.rees.org/apollo-archive/ https://jim.rees.org/apollo-archive/) The answer to the following question was posted by Dave Tweed, a former Apollo engineer who worked on the MMU design for later Apollo workstations (see his resume at http://www.dtweed.com/resume2.html http://www.dtweed.com/resume2.html): "> Hmm. So how did CPU B know that this was needed? It sounds like they > weren't running the same code, after all. No, of course not. The MMU was external to both CPUs, and when one CPU encountered a page fault, the MMU would interrupt the other CPU to handle the exception. The CPUs shared the main memory, so in that sense they were running the same code -- e.g., there was only one copy of the exception handler that either CPU could run. But they were not lock-stepped the way the OP described. I can't even imagine how you'd accomplish that in general." However, from another thread, also by Jim Rees (https://groups.google.com/g/comp.sys.apollo/c/GzEtBHVHEvA/m/7bBmrCurVSoJ https://groups.google.com/g/comp.sys.apollo/c/GzEtBHVHEvA/m/...): "Back in the good old days, the 68000 was the only Motorola processor available. It did not have restartable instructions, so you couldn't do VM with them. Early Suns didn't have VM, they squeezed everything into real memory, like a pdp-11. Early Apollos had an ingenious VM system. There were two processors, the 'A' and the 'B'. One of them would run until it took a page fault, then it would go into a memory wait while the other one brought the page in. It's not really parallel in the usual sense of the word, but there were certain things that could run on either processor, and in fact you could have both of them running at the same time. Normal user processes only ran on one." What a fascinating rabbit hole...

johndoe0815 3y ago

Please note that I can't confirm that Apollo used this approach - the earliest Apollo workstation I know (the DN300) used a 68010 already. I would love to hear more details on this... I'm also not sure which approach early SGI workstations used (the Iris 1000 series), documentation on those is extremely hard to find.

Tor3 3y ago
There's a Motorola document about this: https://ia904706.us.archive.org/5/items/Motorola-SeminarsandApplicationBooksDC-001VirtualMemoryUsingTheMC68000AndTheMC68451MMU/Motorola-SeminarsandApplicationBooksDC-001VirtualMemoryUsingTheMC68000AndTheMC68451MMU_text.pdf https://ia904706.us.archive.org/5/items/Motorola-Seminarsand... In there they suggest that "another bus master must perform this action" and that this one could be "an MC68000 MPU as well". I've also heard and seen the claim that Apollo workstations before the 68010 used this method, but it's difficult to find a definite source for this (though it's likely true. Every reference etc. to the pre-DN300 models claim 2x68000 CPU)
- eschneider 3y ago
  I can confirm that the early Apollos did this. Worked on one back in the day.
ThomasBHickey 3y ago
They did indeed use multiple 68K chips. In fact I have one of the motherboards down in my basement (sadly not operational).

jonstewart 3y ago

This is great info. Can you elaborate or point to a link on the distinction between paging and swapping? I’ve always treated them as synonyms but clearly there are some differences on earlier systems.

johndoe0815 3y ago
Swapping (in the context of old Unix) is always replacing (copying) an entire process address space (or at least the non-shareable read-write segments such as .data), out to the swap device, or back, in one go. Paging is more advanced as it allows to keep only a part of the memory required by a process in RAM and contents can be loaded or moved to secondary storage on the granularity of single pages (typically 4 kB, but larger pages are also common nowadays). So a typical exec(2) system call in Unix could set up a new page table which contains no single byte of the executable in the beginning and then jump to the executable's entry point. This, in turn, results in a page fault, in turn, the OS can then load the requested page into RAM. Paging is more efficient since control/data flow analyses show that something similar to the Pareto rule (or 80/20 rule) also applies to programs which are executed - 80% of the time, only 20% of the address range (for instructions and/or data) is used. With paging, the OS won't need to load any code or data from disk that is never used (and can load it on demand later). What's probably confusing is that the area reserved on disk for storing RAM contents (a partition or a file) is still called "swap partition/file". This is an old term that refuses to die, I guess...
- jonstewart 3y ago
  So “old” swapping is taking low priority processes and putting all of their memory out to disk, almost (but obviously not quite) as though you’re suspending them. Got it!
  - deleted 3y ago
    [deleted]

johndoe0815 3y ago

Here's an interesting discussion thread in comp.sys.apollo from 2003: https://groups.google.com/g/comp.sys.apollo/c/55dqpf11hEw/m/yvrhy2AlZVwJ https://groups.google.com/g/comp.sys.apollo/c/55dqpf11hEw/m/... The setup was more complex than I described, as the second 68000 did not simply wait for a page fault to happen but was executing regular code (unclear if application only or also OS kernel, but the code was not allowed to raise page faults, user processes usually ran on one CPU, see the last quote below) and only handled the fault on the other CPU when interrupted by the MMU. Unfortunately, the only versions of Aegis/DomainOS available online seem to be 10.x (on bitsavers), but support for the "sau1" 68000-based systems (DN100/DN400/DN416) ended with version 9.x, so we can't have a look at the code implementing this feature. Interesting quotes from the thread: "The two processors were called "A" and "B" and processes could be marked runnable on A or on B or both. It was almost a multiprocessor, except that you couldn't run anything that might take a page fault on both at the same time." (by Jim Rees, who runs the Apollo archive at https://jim.rees.org/apollo-archive/ https://jim.rees.org/apollo-archive/) The answer to the following question was posted by Dave Tweed, a former Apollo engineer who worked on the MMU design for later Apollo workstations (see his resume at http://www.dtweed.com/resume2.html http://www.dtweed.com/resume2.html): "> Hmm. So how did CPU B know that this was needed? It sounds like they > weren't running the same code, after all. No, of course not. The MMU was external to both CPUs, and when one CPU encountered a page fault, the MMU would interrupt the other CPU to handle the exception. The CPUs shared the main memory, so in that sense they were running the same code -- e.g., there was only one copy of the exception handler that either CPU could run. But they were not lock-stepped the way the OP described. I can't even imagine how you'd accomplish that in general." However, from another thread, also by Jim Rees (https://groups.google.com/g/comp.sys.apollo/c/GzEtBHVHEvA/m/7bBmrCurVSoJ https://groups.google.com/g/comp.sys.apollo/c/GzEtBHVHEvA/m/...): "Back in the good old days, the 68000 was the only Motorola processor available. It did not have restartable instructions, so you couldn't do VM with them. Early Suns didn't have VM, they squeezed everything into real memory, like a pdp-11. Early Apollos had an ingenious VM system. There were two processors, the 'A' and the 'B'. One of them would run until it took a page fault, then it would go into a memory wait while the other one brought the page in. It's not really parallel in the usual sense of the word, but there were certain things that could run on either processor, and in fact you could have both of them running at the same time. Normal user processes only ran on one." What a fascinating rabbit hole...