3 ms·

Is a Beowulf cluster something conceptually different from any compute cluster / super computer running Linux today?

by rerx 4y ago

trelane 4y ago
I think the main distinction is that modern clusters are shiny, sleek, fancy, and expensive. Whereas a Beowulf cluster is grungy, full of bailing twine, chewing gum, and possessing high levels of Bodge. And dirt cheap, built with what you can get, often used.
icco 4y ago
Beowulfs were huge hacks back in the day, but are similar to most super computers. The main difference from how many think of compute clusters these days is it is managed as a single machine, so historically needed identical hardware. Parallel processing and concurrency used to be more rare :D https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster
dekhn 4y ago
The defining attribute of original Beowulf clusters that made them interesting was the use of channel bonding to get more bandwidth than a single NIC could provide cheaply in those days. At the same time, it demonstrated you could build a useful and inexpensive cluster using white-box linux desktop machines sitting on wire shelving. I did exactly the same thing at the time, except for the channel bonding, which would have helped my cluster scaling significantly, but I didn't have any budget left over for more NICs and since I was using a hub, there would have been lots of collisions anyway.
retrac 4y ago
It's a cluster, made out of commodity computers and network equipment, that wasn't designed specifically to be part of a cluster. "Beowulf" was just the name of an early machine built that way at NASA in the 90s. The approach is not unusual now; many "low-end" supercomputers are just a few racks of commodity servers. But it was very unconventional in the early 90s, which is why the Beowulf term existed, to distinguish it from "real" clusters. Hardware Ethernet switches -- able to stream from port to port at full speed -- were the revolutionary component. Before Ethernet, clusters required specialized hardware implementing a crossbar-like switched fabric, so that all the nodes could communicate at high speed with each other, with a minimum of hops. These were solutions for clustering mainframes and minicomputers, horribly expensive and proprietary.

Is a Beowulf cluster something conceptually different from any compute cluster / super computer running Linux today?

by rerx 4y ago

Is a Beowulf cluster something conceptually different from any compute cluster / super computer running Linux today?

I think the main distinction is that modern clusters are shiny, sleek, fancy, and expensive. Whereas a Beowulf cluster is grungy, full of bailing twine, chewing gum, and possessing high levels of Bodge. And dirt cheap, built with what you can get, often used.

icco 4y ago

Beowulfs were huge hacks back in the day, but are similar to most super computers. The main difference from how many think of compute clusters these days is it is managed as a single machine, so historically needed identical hardware. Parallel processing and concurrency used to be more rare :D https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster

dekhn 4y ago

The defining attribute of original Beowulf clusters that made them interesting was the use of channel bonding to get more bandwidth than a single NIC could provide cheaply in those days. At the same time, it demonstrated you could build a useful and inexpensive cluster using white-box linux desktop machines sitting on wire shelving. I did exactly the same thing at the time, except for the channel bonding, which would have helped my cluster scaling significantly, but I didn't have any budget left over for more NICs and since I was using a hub, there would have been lots of collisions anyway.

retrac 4y ago

It's a cluster, made out of commodity computers and network equipment, that wasn't designed specifically to be part of a cluster. "Beowulf" was just the name of an early machine built that way at NASA in the 90s. The approach is not unusual now; many "low-end" supercomputers are just a few racks of commodity servers. But it was very unconventional in the early 90s, which is why the Beowulf term existed, to distinguish it from "real" clusters. Hardware Ethernet switches -- able to stream from port to port at full speed -- were the revolutionary component. Before Ethernet, clusters required specialized hardware implementing a crossbar-like switched fabric, so that all the nodes could communicate at high speed with each other, with a minimum of hops. These were solutions for clustering mainframes and minicomputers, horribly expensive and proprietary.