4 ms·

What were the compute cells like in the Connection Machine? The article uses the phrase "single-bit processor", but that doesn't sound all that useful - you'd

by chiph 13y ago

What were the compute cells like in the Connection Machine? The article uses the phrase "single-bit processor", but that doesn't sound all that useful - you'd want something with at least a little memory and a decent instruction set. Too small a cell and it runs out of work and spends most of it's time waiting on the network. Too big and you'll run into packaging problems.

hga 13y ago
From old memory (of Danny Hillis explaining it in a general seminar, etc.; if they'd had their hiring act together, I would have joined them), the key in the CM-1 is the connections. It was a SIMD machine, a single instruction at a time was transmitted to all the CPUs (one of the reasons a company had to be formed, the top notch analog EEs he had to hire to make it work had salaries higher than an MIT professor's, which just isn't done), and depending on their state they'd do something, or perhaps ignore it. How you laid out your data was key; what I remember of the example he gave, each street intersection was allocated to a CPU. There's of course much better accounts out there, but this might give you a flavor. It's entirely unlike the usual MIMD machines which they and many others built later.
- csense 13y ago
  The wikipedia article for the Connection Machine has a link to manuals. The one I'm reading is for the CM-5, so maybe the earlier arches were different, but to me it looks an awful lot like modern GPU's. [1] http://people.csail.mit.edu/bradley/cm5docs/ http://people.csail.mit.edu/bradley/cm5docs/
  - abecedarius 13y ago
    Yes, the CM-5 was very different. You want the CM-1 or 2.
EdwardCoffin 13y ago
Guy Steele talked briefly about this in an interview with Dr. Dobb's magazine [1]. Here's the relevant excerpt: As my leave of absence was ending, I was still involved at Carnegie Mellon and showed up for a seminar given by Danny Hillis [http://www.edge.org/digerati/hillis/ http://www.edge.org/digerati/hillis/] about the CM-1 Connection Machine [http://mission.base.com/tamiko/cm/ http://mission.base.com/tamiko/cm/]. He went on quite enthusiastically about artificial intelligence and neural networks but thought the CM-1 was less suited for numerical programming. Whenever you tell me that you can't make a computer do something, it's a red flag. I believe the Turing hypothesis. The only question is the efficiency of the calculation. I believe in crunching the numbers, doing the calculations, and finding out whether an assertion is true or not. So while Danny finished the lecture, I scribbled calculations on the back of an envelope. Okay, suppose you did the arithmetic bit serially, like the PDP-8S? A 32-bit floating-point multiply you have to multiply two 24-bit significands, that's going to take 600 time steps. For the additions, a logarithmic number of shifts, each will take 24 time steps...It turned out that a floating-point add would take about 1/2 a millisecond and a multiply a millisecond. Sounds terrible, but you've got 64,000 processors. DDJ: So you could do 64,000 of these calculations in a millisecond. GS: That's up in the range of 100 million operations a second, equivalent to a Cray 1, which was the supercomputer of the day. So after the lecture, I went up to Danny and said, "You said this is a terrible numeric machine, but I can tell it's approximately the equivalent of a Cray 1." And he replied, "Can we have dinner tonight?" [1] http://www.drdobbs.com/jvm/a-conversation-with-guy-steele-jr/184406029 http://www.drdobbs.com/jvm/a-conversation-with-guy-steele-jr...
- greenyoda 13y ago
  Being able to theoretically do 100 million operations per second doesn't make a machine competitive with a Cray 1. You also have to consider stuff like the memory and I/O bandwidth. If you can only get 1000 floating point numbers into memory in a second, being able to run at 100 megaflops doesn't help much. Also, you can't really use all those processors unless you have an algorithm that's massively parallel. Any time your algorithm hits a piece of code that's sequential rather than parallel, you've gone from a Cray 1 back to a PDP-8.
  - EdwardCoffin 13y ago
    The fundamental assumption they were making, though, was that they would be using parallel algorithms. They intended to do things like finite-element analysis where each element would be pinned to a particular processor, and would only need to communicate with neighbours, which was exactly the kind of thing this architecture was designed to do.
rst 13y ago
The processors on the CM-1 were indeed bit-serial, with 16 of these per processor chip. On the CM-2, pairs of 16-processor chips were lashed up with a Weitek FPU and glue circuitry for floating point computation. For the CM-5, processor nodes were Sparc chips with custom vector processors.
jloughry 13y ago
This is my favourite Connection Machine story: The Connection Machine began as Hillis's Ph.D. thesis at MIT. "There was a point when Danny was getting serious about the design," recalls Minsky. "And I said to him, 'I hope you don't make the ILLIAC IV mistake.' And he said, 'Oh, what's the ILLIAC IV mistake?' And I told him." . . . "About a month later," Minsky recalls, "Danny came back to me and said, 'Well, I've decided to make the ILLIAC IV mistake." [1, pp. 203--204] [1] From the book Archimedes' Revenge: The Joys and Perils of Mathematics by Paul Hoffman (New York: Ballantine Books, 1988).
- marshray 13y ago
  What was the ILLIAC IV mistake? Does it say?
  - jloughry 13y ago
    The book is out of print and difficult to find, so I reproduce an extract here; I hope this is Fair Use: The ILLIAC IV was a gigantic machine built in the 1970s at the University of Illinois. It had sixty-four processors, each as big as an upright piano because it was built before its time. In fact, they had to use a forklift to plug in the units. It took seven or eight years to build, and the machine was obsolete by the time it was finished. The university gave it to NASA, which promised to use it but had difficulty doing so. That the project was too ambitious, given the available technology, was not what Minsky meant by the ILLIAC IV mistake. He told Hillis that the concept itself was flawed; it was a mistake, he said, to restrict all the sixty-four processors to doing exactly the same thing at the same time, an expanded electronic version of the Olympic event synchronized swimming. Minsky told Hillis that the processors should be able to operate independently. "About a month later," Minsky recalls, "Danny came back to me and said, 'Well, I've decided to make the ILLIAC IV mistake.'" Hillis told Minsky that the problem was not in what the processors did but in how they communicated. He said that intraprocessor signals got stuck in traffic jams because, among other things, the connecting wires were restricted to two dimensions. Hillis recognized that a richer connection scheme was required, all the more so since he wanted to hook up not 64 processors but 65,536. The signal traffic would be equivalent to a telephone network serving 65,536 customers who placed a quarter of a billion calls per second. That was the chief technical problem Hillis faced in designing the Connection Machine. In the end, Hillis and his co-workers at Thinking Machines settled on a three-dimensional architecture in which the processors were connected as if they formed a sixteen-dimensional cube. What this means is that each processor, although directly connected to only 16 others, was never more than sixteen steps away from any of the other 65,536 processors. Moreover, the possibility of a traffic jam was reduced because in sixteen dimensions there are numerous possible routes between any two processors. [1] [1] From the book Archimedes' Revenge: The Joys and Perils of Mathematics by Paul Hoffman (New York: Ballantine Books, 1988), pp. 203--4.
mark_l_watson 13y ago
I wrote application software for a CM-1. It was a SIMD machine. Later models were MIMD. The programming language I used Lisp* (star lisp) was pretty neat.

by chiph 13y ago

hga 13y ago

From old memory (of Danny Hillis explaining it in a general seminar, etc.; if they'd had their hiring act together, I would have joined them), the key in the CM-1 is the connections. It was a SIMD machine, a single instruction at a time was transmitted to all the CPUs (one of the reasons a company had to be formed, the top notch analog EEs he had to hire to make it work had salaries higher than an MIT professor's, which just isn't done), and depending on their state they'd do something, or perhaps ignore it. How you laid out your data was key; what I remember of the example he gave, each street intersection was allocated to a CPU. There's of course much better accounts out there, but this might give you a flavor. It's entirely unlike the usual MIMD machines which they and many others built later.

csense 13y ago
The wikipedia article for the Connection Machine has a link to manuals. The one I'm reading is for the CM-5, so maybe the earlier arches were different, but to me it looks an awful lot like modern GPU's. [1] http://people.csail.mit.edu/bradley/cm5docs/ http://people.csail.mit.edu/bradley/cm5docs/
- abecedarius 13y ago
  Yes, the CM-5 was very different. You want the CM-1 or 2.

EdwardCoffin 13y ago

Guy Steele talked briefly about this in an interview with Dr. Dobb's magazine [1]. Here's the relevant excerpt: As my leave of absence was ending, I was still involved at Carnegie Mellon and showed up for a seminar given by Danny Hillis [http://www.edge.org/digerati/hillis/ http://www.edge.org/digerati/hillis/] about the CM-1 Connection Machine [http://mission.base.com/tamiko/cm/ http://mission.base.com/tamiko/cm/]. He went on quite enthusiastically about artificial intelligence and neural networks but thought the CM-1 was less suited for numerical programming. Whenever you tell me that you can't make a computer do something, it's a red flag. I believe the Turing hypothesis. The only question is the efficiency of the calculation. I believe in crunching the numbers, doing the calculations, and finding out whether an assertion is true or not. So while Danny finished the lecture, I scribbled calculations on the back of an envelope. Okay, suppose you did the arithmetic bit serially, like the PDP-8S? A 32-bit floating-point multiply you have to multiply two 24-bit significands, that's going to take 600 time steps. For the additions, a logarithmic number of shifts, each will take 24 time steps...It turned out that a floating-point add would take about 1/2 a millisecond and a multiply a millisecond. Sounds terrible, but you've got 64,000 processors. DDJ: So you could do 64,000 of these calculations in a millisecond. GS: That's up in the range of 100 million operations a second, equivalent to a Cray 1, which was the supercomputer of the day. So after the lecture, I went up to Danny and said, "You said this is a terrible numeric machine, but I can tell it's approximately the equivalent of a Cray 1." And he replied, "Can we have dinner tonight?" [1] http://www.drdobbs.com/jvm/a-conversation-with-guy-steele-jr/184406029 http://www.drdobbs.com/jvm/a-conversation-with-guy-steele-jr...

greenyoda 13y ago
Being able to theoretically do 100 million operations per second doesn't make a machine competitive with a Cray 1. You also have to consider stuff like the memory and I/O bandwidth. If you can only get 1000 floating point numbers into memory in a second, being able to run at 100 megaflops doesn't help much. Also, you can't really use all those processors unless you have an algorithm that's massively parallel. Any time your algorithm hits a piece of code that's sequential rather than parallel, you've gone from a Cray 1 back to a PDP-8.
- EdwardCoffin 13y ago
  The fundamental assumption they were making, though, was that they would be using parallel algorithms. They intended to do things like finite-element analysis where each element would be pinned to a particular processor, and would only need to communicate with neighbours, which was exactly the kind of thing this architecture was designed to do.

rst 13y ago

The processors on the CM-1 were indeed bit-serial, with 16 of these per processor chip. On the CM-2, pairs of 16-processor chips were lashed up with a Weitek FPU and glue circuitry for floating point computation. For the CM-5, processor nodes were Sparc chips with custom vector processors.

jloughry 13y ago

This is my favourite Connection Machine story: The Connection Machine began as Hillis's Ph.D. thesis at MIT. "There was a point when Danny was getting serious about the design," recalls Minsky. "And I said to him, 'I hope you don't make the ILLIAC IV mistake.' And he said, 'Oh, what's the ILLIAC IV mistake?' And I told him." . . . "About a month later," Minsky recalls, "Danny came back to me and said, 'Well, I've decided to make the ILLIAC IV mistake." [1, pp. 203--204] [1] From the book Archimedes' Revenge: The Joys and Perils of Mathematics by Paul Hoffman (New York: Ballantine Books, 1988).

marshray 13y ago
What was the ILLIAC IV mistake? Does it say?
- jloughry 13y ago
  The book is out of print and difficult to find, so I reproduce an extract here; I hope this is Fair Use: The ILLIAC IV was a gigantic machine built in the 1970s at the University of Illinois. It had sixty-four processors, each as big as an upright piano because it was built before its time. In fact, they had to use a forklift to plug in the units. It took seven or eight years to build, and the machine was obsolete by the time it was finished. The university gave it to NASA, which promised to use it but had difficulty doing so. That the project was too ambitious, given the available technology, was not what Minsky meant by the ILLIAC IV mistake. He told Hillis that the concept itself was flawed; it was a mistake, he said, to restrict all the sixty-four processors to doing exactly the same thing at the same time, an expanded electronic version of the Olympic event synchronized swimming. Minsky told Hillis that the processors should be able to operate independently. "About a month later," Minsky recalls, "Danny came back to me and said, 'Well, I've decided to make the ILLIAC IV mistake.'" Hillis told Minsky that the problem was not in what the processors did but in how they communicated. He said that intraprocessor signals got stuck in traffic jams because, among other things, the connecting wires were restricted to two dimensions. Hillis recognized that a richer connection scheme was required, all the more so since he wanted to hook up not 64 processors but 65,536. The signal traffic would be equivalent to a telephone network serving 65,536 customers who placed a quarter of a billion calls per second. That was the chief technical problem Hillis faced in designing the Connection Machine. In the end, Hillis and his co-workers at Thinking Machines settled on a three-dimensional architecture in which the processors were connected as if they formed a sixteen-dimensional cube. What this means is that each processor, although directly connected to only 16 others, was never more than sixteen steps away from any of the other 65,536 processors. Moreover, the possibility of a traffic jam was reduced because in sixteen dimensions there are numerous possible routes between any two processors. [1] [1] From the book Archimedes' Revenge: The Joys and Perils of Mathematics by Paul Hoffman (New York: Ballantine Books, 1988), pp. 203--4.

mark_l_watson 13y ago

I wrote application software for a CM-1. It was a SIMD machine. Later models were MIMD. The programming language I used Lisp* (star lisp) was pretty neat.