4 ms·

Serious question, are Beowulf clusters still a thing?

by ty_a 9y ago

zrail 9y ago
Sure. They're not really called Beowulf clusters anymore, but even at the beginning there wasn't any particular defining characteristic other than a cluster of commodity servers running some kind of software that made them work together. You could consider the clusters at LHC, the render farms at Weta and Pixar, and (if you squint) the non-web clusters at Google and Facebook to all be spiritual descendants of the original Beowulf cluster. https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster
- GauntletWizard 9y ago
  Huh. I always thought that Beowulf was an actual piece of software. But apparently it's only a capability set, and by its definition every Mesos and Kubernetes installation counts. Cool.
- pjmlp 9y ago
  Ah, still remember the early days of the LHC Computing Grid.....
- bogomipz 9y ago
  >"... the non-web clusters at Google and Facebook to all be spiritual descendants of the original Beowulf cluster." I was intrigued by this comment. Could you elaborate on this? Are there specific concepts from Beowulf that FB's Tupperware and Google's Borg borrowed?
  - zrail 9y ago
    The entirety of the Beowulf concept is a cluster of commodity machines networked together that can work together to perform some kind of task. It's pretty general. The original Beowulf-style clusters ran things like mosix to present a single system image for applications but as far as I can tell that was never a defining characteristic.
    - bogomipz 9y ago
      Thanks, yes that's pretty broad, makes sense. Cheers.
rbanffy 9y ago
Sometimes a very little thing ;-) https://shop.pimoroni.com/products/cluster-hat https://shop.pimoroni.com/products/cluster-hat
- sannee 9y ago
  This shows a pet peeve of mine: the Raspberry Pi does not have USB available in a way which would allow a HAT to interface with it in a more sensible way than wrapping around a cable from the "external" ports. I wonder if there is a pair of 90 degree USB connectors which would height-match a standard .1" header.
  - topspin 9y ago
    Perhaps the "Compute Module" is the thing to use for Raspberry Pi clusters. Probably fit 100+ in 1U with no cables at all.
    - rbanffy 9y ago
      I'd love to see a revision/extension of the Pi connector adding the Ethernet signalling. This would make it easy to build clusters using any board compatible with the ecosystem. The SOPINE (a Pine A64 cousin https://www.pine64.org/?page_id=1491 https://www.pine64.org/?page_id=1491) seems to have the Ethernet signals on the edge connector, making a cluster much easier to build out of a single backplane. The Rock64, an A64 successor, has the Ethernet on pins, but the position is a bit awkward. But yes. At least the USB should be available on the expansion bus.

Serious question, are Beowulf clusters still a thing?

by ty_a 9y ago

Serious question, are Beowulf clusters still a thing?

Sure. They're not really called Beowulf clusters anymore, but even at the beginning there wasn't any particular defining characteristic other than a cluster of commodity servers running some kind of software that made them work together. You could consider the clusters at LHC, the render farms at Weta and Pixar, and (if you squint) the non-web clusters at Google and Facebook to all be spiritual descendants of the original Beowulf cluster. https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster

GauntletWizard 9y ago
Huh. I always thought that Beowulf was an actual piece of software. But apparently it's only a capability set, and by its definition every Mesos and Kubernetes installation counts. Cool.
pjmlp 9y ago
Ah, still remember the early days of the LHC Computing Grid.....
bogomipz 9y ago
>"... the non-web clusters at Google and Facebook to all be spiritual descendants of the original Beowulf cluster." I was intrigued by this comment. Could you elaborate on this? Are there specific concepts from Beowulf that FB's Tupperware and Google's Borg borrowed?
- zrail 9y ago
  The entirety of the Beowulf concept is a cluster of commodity machines networked together that can work together to perform some kind of task. It's pretty general. The original Beowulf-style clusters ran things like mosix to present a single system image for applications but as far as I can tell that was never a defining characteristic.
  - bogomipz 9y ago
    Thanks, yes that's pretty broad, makes sense. Cheers.

rbanffy 9y ago

Sometimes a very little thing ;-) https://shop.pimoroni.com/products/cluster-hat https://shop.pimoroni.com/products/cluster-hat

sannee 9y ago
This shows a pet peeve of mine: the Raspberry Pi does not have USB available in a way which would allow a HAT to interface with it in a more sensible way than wrapping around a cable from the "external" ports. I wonder if there is a pair of 90 degree USB connectors which would height-match a standard .1" header.
- topspin 9y ago
  Perhaps the "Compute Module" is the thing to use for Raspberry Pi clusters. Probably fit 100+ in 1U with no cables at all.
  - rbanffy 9y ago
    I'd love to see a revision/extension of the Pi connector adding the Ethernet signalling. This would make it easy to build clusters using any board compatible with the ecosystem. The SOPINE (a Pine A64 cousin https://www.pine64.org/?page_id=1491 https://www.pine64.org/?page_id=1491) seems to have the Ethernet signals on the edge connector, making a cluster much easier to build out of a single backplane. The Rock64, an A64 successor, has the Ethernet on pins, but the position is a bit awkward. But yes. At least the USB should be available on the expansion bus.