3 ms·

I'll try to reply to this in sections. One way to think of "magic" is height of leap. Newton had something before him, but his leap was cosmic. I don't think we

I'll try to reply to this in sections. One way to think of "magic" is height of leap. Newton had something before him, but his leap was cosmic. I don't think we've had a Newton yet in computing, but we have had some cosmic leaps. Just to pick a few: Sketchpad, Lisp, and the B5000 ideas all had something before them, but their leaps were enormous. In the "Early History of Smalltalk" I mentioned that I had seen and partly learned the B5000 while in the Air Force ca 1963. I say "partly" because I didn't understand all the implications until a few years later in grad school. For example, I did understand what was great about the byte codes and stack mechanisms, but it took until later to really grok that the machine was organized so that environments and access were -granted-, and that the protected tagged pointers -- Descriptors -- could not be forged, and that the actual code contained no addresses. (This was the real start of "capability-style protection.) Also, the multi-tasking OS -- written in an Extended Algol -- was not just the first OS in a higher level language, but was more profound than Algol (it was "Simula" before Simula). This was done extremely well because -- and most people don't realize this -- the B5000 had multiple CPUs! More in a next note ...

alankay1 10y ago
I've been a huge fan of Margaret Hamilton for a long time. Her story is in many ways what can be wonderful about great design and creation by really smart and able people, and so disappointing about our not-quite-a-field. (And if you want a tough computer to code and make a great system for, look no further than the Apollo Guidance Computer!) I think that a mainframe computer like the B5000 (or more conventional mainframe) was just too much machinery back then to overlap much with the needs of Apollo). Hamilton realized that a really good notion of "module" would help tremendously with integrity, and that the OS should have an active "overlord" to dynamically assess and deal with real-time conditions and needs for resources. I think this was just amazingly brilliant work for that time (and any time). NASA did give her a major award (but where was the ACM?). I've never had the pleasure of talking with her, but I have a feeling that NASA could also have done something to encourage her ideas to be more communicated and put out into the open.
- alankay1 10y ago
  "Flex" has been a very popular label for computer systems. I was referring to an early desktop computer done by Ed Cheadle and myself in the late 60s for a company owned by LTV (Ling Tempco Vought -- an aerospace conglomerate). The one you are referring to was done in the 80s in the UK (I think). I recall that there were many good things about this system, and it did use microcode in a rather similar way to Parc in the 70s. Wasn't one of the machines they used a PERQ (which was an Alto spinoff by some CMU folks)? I think I recall -- as with the Intel 432 -- they bit off more than they could chew, and tried to optimize too many things.
  - alankay1 10y ago
    The final part is about "microcode" etc. in being able to get the most flexibility/performance from hardware. A good heuristic is to look far out (30 years) for "It would be ridiculous if we didn't have ...". See what any of those might mean 10-15 years out. Simulate the promising ones of these today using $ to pay for what Moore's Law and other engineering will provide at lower cost in the future. This will give a platform today on which the software of the future can be invented, developed, and tested. (This is what we did to get the Alto at Parc -- and this is what it was -- in the mid 70s an Alto cost about $22K -- about $120K today. But it allowed the software of the future to be invented.) Microcode (invented long long ago by Maurice Wilkes, who did the EDSAC, arguably the first real programmable computer) used the argument that if you can make a small amount of memory plus CPU machinery be much faster than main memory, then you can successfully program "machine-level" functionality as though it were just hardware. For example, the Alto could execute about 5 microinstructions for every main memory cycle -- this allowed us to make emulators that were "as fast as they could be". This fit in well with the nature, speed, capacity, etc of the memory available at the time. But "life is not linear", so we have to look around carefully each time we set out to design something. As Butler Lampson has pointed out, one of the things that make good systems design very difficult is that the exponentials involved every few years mean that major design rules may not still obtain just a few years later. So, I would point you here to FPGAs and their current capacities, especially for comingling processing and memory elements (they are the same) in highly parallel architectures. Chuck Thacker, who was mainly responsible for most of the hardware (and more) at Parc, did the world a service by designing the BEE-3 as "an Alto for today" in the form of a number of large FPGA chips plus other goodies. Very worth looking at! The basic principle here is that "Hardware is just software crystallized early" so it's always good to start off with what is essentially a pie in the sky software architecture, and then start trying to see the best way to run this in a particular day and time.
nickpsecurity 10y ago
The nesting gets weird on this site. So, I'm replying to those all at once. re Burroughs. Yeah, I guess there was some OOP principles in its combo of descriptors and modules. Didn't think about that. Many CPU's I knew about. Another thing was how you could pause processes on exception to correct or properly terminate them. Something LISP machines would brag about later on. Something I still want today in such a clean form. re Hamilton. Appreciate confirmation on her skill. Yeah, AGC was some serious constraints. She didn't just do all that stuff in Wikipedia. Her follow-up was essentially inventing or commercializing correct-by-construction with HOL and 001 Toolkit work. Despite the usability & performance drawbacks, it was essentially model-driven development with mathematical notation plus semi-automated design plus automated code, test, and trace generation. Reminiscient of CASE tools but more rigorous & enforced resource use (eg real-time). Showed up in mid-70's. Dijkstra and Hoare's methods are what high-assurance designs used a bit after Hamilton's time in 80's-90's. Methods like Hamilton's are what high-assurance is developing now under DARPA and Euro contracts with subsets like Esterel SCADE in production. So, given inventing most key practices from scratch, I've been calling her the effective founder of software engineering and pioneer in high-assurance that was 10-30 years ahead depending on capability. Deserves way more credit than IT field gives her. NASA at least gave her official recognition plus largest, financial award per 2003 report. re Flex. Thanks for the clarification. Yeah, the big Flex does seem to have made mistake Intel i432 did. Might be a limit to how much of a system can go into hardware with cost & performance efficiency. Still open problem. re microcode. Never knew they mainly were trying to beat the RAM speeds. Gonna have to think on that as fighting memory speeds is a current worry in exascale presentations. The emulator advantage might be long-term esp for backward compatibility for existing or future systems. Worked wonders for AS/400's evolution. re BEE3. Thanks for the tip! On concept level, I was working on an idea for something like this for virtual silicon to reduce number of prototyping runs, esp at lowest nodes. I was going to use instrumented FPGA's or something similar that also capture process-specific stuff. This might have valuable lessons. re design for future. This one I have a hard time getting. For specific tech rather than concepts which make plenty sense. Reason being, it sounds like great advice for about the whole ride through Moore's law. Except most think that's over. So, we may have to modify your recommendation. Right now, they're casting a wide net with proposals + prototypes ranging from 2.5D-3D dies, massive multicore, low multicore + many HW accelerators, expanded GPU's, hybrid CPU/FPGA's, mixed signal (i.e. more analog), and so on. Landscape was homogenous for software where serial or somewhat parallel stuff just gets faster and cheaper. Now, it's so heterogenous that committing to one requires design that might kill success on many others with no clear winner in sight. Just a few likely survivors like Intel, GPU, offload, and FPGA approaches. So, that poses a huge problem when I try to bet on the future for specific, HW tech. Here's some heuristics I have. It seems clear we keep things modular & interface in case HW accelerators show up for any function. Minimizing state or tight coupling is good as concurrency or caching issues will likely stay. Need to map design to coarse- and fine-grained parallelism, configurable or synthesizable, if possible since victor isn't decided. Design for scaling. Minimize effect of failures of individual components as higher transistors & cores = higher failures. Seemless detection and recovery. Protection mechanisms or about anything else still needs to be small and/or parallelizable to get high, clock speed. Much more research on cache design & optimization since it stays an issue. Continue focus on high-level languages that auto-generate to efficient, low-level implementations with extra focus on parallel/concurrent/hardware-friendly synthesis. Those are off top of my head. I'm sure you've noticed the diversity and fragmentation of various HW and computing approaches happening. Have you thought about what to focus on or how to map the software to many emerging tech at once to minimize risk? I'm always up for opinions on it given we're years into exascale & post-Moore research with still no consensus far as I can tell as an outsider to HPC research.

alankay1 10y ago

I've been a huge fan of Margaret Hamilton for a long time. Her story is in many ways what can be wonderful about great design and creation by really smart and able people, and so disappointing about our not-quite-a-field. (And if you want a tough computer to code and make a great system for, look no further than the Apollo Guidance Computer!) I think that a mainframe computer like the B5000 (or more conventional mainframe) was just too much machinery back then to overlap much with the needs of Apollo). Hamilton realized that a really good notion of "module" would help tremendously with integrity, and that the OS should have an active "overlord" to dynamically assess and deal with real-time conditions and needs for resources. I think this was just amazingly brilliant work for that time (and any time). NASA did give her a major award (but where was the ACM?). I've never had the pleasure of talking with her, but I have a feeling that NASA could also have done something to encourage her ideas to be more communicated and put out into the open.

alankay1 10y ago
"Flex" has been a very popular label for computer systems. I was referring to an early desktop computer done by Ed Cheadle and myself in the late 60s for a company owned by LTV (Ling Tempco Vought -- an aerospace conglomerate). The one you are referring to was done in the 80s in the UK (I think). I recall that there were many good things about this system, and it did use microcode in a rather similar way to Parc in the 70s. Wasn't one of the machines they used a PERQ (which was an Alto spinoff by some CMU folks)? I think I recall -- as with the Intel 432 -- they bit off more than they could chew, and tried to optimize too many things.
- alankay1 10y ago
  The final part is about "microcode" etc. in being able to get the most flexibility/performance from hardware. A good heuristic is to look far out (30 years) for "It would be ridiculous if we didn't have ...". See what any of those might mean 10-15 years out. Simulate the promising ones of these today using $ to pay for what Moore's Law and other engineering will provide at lower cost in the future. This will give a platform today on which the software of the future can be invented, developed, and tested. (This is what we did to get the Alto at Parc -- and this is what it was -- in the mid 70s an Alto cost about $22K -- about $120K today. But it allowed the software of the future to be invented.) Microcode (invented long long ago by Maurice Wilkes, who did the EDSAC, arguably the first real programmable computer) used the argument that if you can make a small amount of memory plus CPU machinery be much faster than main memory, then you can successfully program "machine-level" functionality as though it were just hardware. For example, the Alto could execute about 5 microinstructions for every main memory cycle -- this allowed us to make emulators that were "as fast as they could be". This fit in well with the nature, speed, capacity, etc of the memory available at the time. But "life is not linear", so we have to look around carefully each time we set out to design something. As Butler Lampson has pointed out, one of the things that make good systems design very difficult is that the exponentials involved every few years mean that major design rules may not still obtain just a few years later. So, I would point you here to FPGAs and their current capacities, especially for comingling processing and memory elements (they are the same) in highly parallel architectures. Chuck Thacker, who was mainly responsible for most of the hardware (and more) at Parc, did the world a service by designing the BEE-3 as "an Alto for today" in the form of a number of large FPGA chips plus other goodies. Very worth looking at! The basic principle here is that "Hardware is just software crystallized early" so it's always good to start off with what is essentially a pie in the sky software architecture, and then start trying to see the best way to run this in a particular day and time.

nickpsecurity 10y ago

The nesting gets weird on this site. So, I'm replying to those all at once. re Burroughs. Yeah, I guess there was some OOP principles in its combo of descriptors and modules. Didn't think about that. Many CPU's I knew about. Another thing was how you could pause processes on exception to correct or properly terminate them. Something LISP machines would brag about later on. Something I still want today in such a clean form. re Hamilton. Appreciate confirmation on her skill. Yeah, AGC was some serious constraints. She didn't just do all that stuff in Wikipedia. Her follow-up was essentially inventing or commercializing correct-by-construction with HOL and 001 Toolkit work. Despite the usability & performance drawbacks, it was essentially model-driven development with mathematical notation plus semi-automated design plus automated code, test, and trace generation. Reminiscient of CASE tools but more rigorous & enforced resource use (eg real-time). Showed up in mid-70's. Dijkstra and Hoare's methods are what high-assurance designs used a bit after Hamilton's time in 80's-90's. Methods like Hamilton's are what high-assurance is developing now under DARPA and Euro contracts with subsets like Esterel SCADE in production. So, given inventing most key practices from scratch, I've been calling her the effective founder of software engineering and pioneer in high-assurance that was 10-30 years ahead depending on capability. Deserves way more credit than IT field gives her. NASA at least gave her official recognition plus largest, financial award per 2003 report. re Flex. Thanks for the clarification. Yeah, the big Flex does seem to have made mistake Intel i432 did. Might be a limit to how much of a system can go into hardware with cost & performance efficiency. Still open problem. re microcode. Never knew they mainly were trying to beat the RAM speeds. Gonna have to think on that as fighting memory speeds is a current worry in exascale presentations. The emulator advantage might be long-term esp for backward compatibility for existing or future systems. Worked wonders for AS/400's evolution. re BEE3. Thanks for the tip! On concept level, I was working on an idea for something like this for virtual silicon to reduce number of prototyping runs, esp at lowest nodes. I was going to use instrumented FPGA's or something similar that also capture process-specific stuff. This might have valuable lessons. re design for future. This one I have a hard time getting. For specific tech rather than concepts which make plenty sense. Reason being, it sounds like great advice for about the whole ride through Moore's law. Except most think that's over. So, we may have to modify your recommendation. Right now, they're casting a wide net with proposals + prototypes ranging from 2.5D-3D dies, massive multicore, low multicore + many HW accelerators, expanded GPU's, hybrid CPU/FPGA's, mixed signal (i.e. more analog), and so on. Landscape was homogenous for software where serial or somewhat parallel stuff just gets faster and cheaper. Now, it's so heterogenous that committing to one requires design that might kill success on many others with no clear winner in sight. Just a few likely survivors like Intel, GPU, offload, and FPGA approaches. So, that poses a huge problem when I try to bet on the future for specific, HW tech. Here's some heuristics I have. It seems clear we keep things modular & interface in case HW accelerators show up for any function. Minimizing state or tight coupling is good as concurrency or caching issues will likely stay. Need to map design to coarse- and fine-grained parallelism, configurable or synthesizable, if possible since victor isn't decided. Design for scaling. Minimize effect of failures of individual components as higher transistors & cores = higher failures. Seemless detection and recovery. Protection mechanisms or about anything else still needs to be small and/or parallelizable to get high, clock speed. Much more research on cache design & optimization since it stays an issue. Continue focus on high-level languages that auto-generate to efficient, low-level implementations with extra focus on parallel/concurrent/hardware-friendly synthesis. Those are off top of my head. I'm sure you've noticed the diversity and fragmentation of various HW and computing approaches happening. Have you thought about what to focus on or how to map the software to many emerging tech at once to minimize risk? I'm always up for opinions on it given we're years into exascale & post-Moore research with still no consensus far as I can tell as an outsider to HPC research.