5 ms·

I think this is cool, but engineering for engineering's sake aside I've seriously looked at building a raspi cluster a couple of times and have always concluded

by djaque 7y ago

I think this is cool, but engineering for engineering's sake aside I've seriously looked at building a raspi cluster a couple of times and have always concluded that it isn't worth it. The real selling point for me is energy efficiency and price, but the reason I could never make it happen is because most of the applications I want to use are x86 only and I don't have the resources to be a trailblazer in that area. In addition sourcing becomes an issue with raspberry pis when you aren't a big company like Oracle. It's hard to buy more than a handful as an individual.

jacquesm 7y ago
The real selling point for me would be physical isolation. If you run all of them single-core you wouldn't have to worry at all about snooping using the various side channel attacks.
- Skunkleton 7y ago
  Yes, but how many RPis do you need to match the workload of a single x86 core? How much does it cost to cluster and maintain these systems? IMO you would probably have a better story all around if you based your cluster on low end AMD chips, or perhaps even old refurbished systems. That's not to say this isn't cool engineering for engineering's sake. Just that if you are considering practical applications for this, maybe think twice.
  - jacquesm 7y ago
    When you need that guarantee of isolation you are picking your solution for different reasons than when you need a particular workload. So there is no point comparing the workload if that wasn't the deciding factor in the first place.
    - Skunkleton 7y ago
      My point is that you can probably get physical isolation for less money on x86.
      - jacquesm 7y ago
        You're going to have to explain a bit more on how you intend to achieve that economy. I'm not aware of a single SOC+peripherals based on x86 that would beat the Pi in price/performance, is there such a thing?
willis936 7y ago
In case people reading play with raspis and aren’t aware: you can reduce power consumption by several percent by disabling peripherals. This wiki has some useful information for those looking to make raspi clusters. https://www.pidramble.com/wiki/benchmarks/power-consumption https://www.pidramble.com/wiki/benchmarks/power-consumption
giancarlostoro 7y ago
The Raspberry Pi Zero's are ridiculously affordable. For the cost of a regular Pi 3 you can get one thats wireless with a case with 3 covers for the case (1 for the Raspberry Pi camera). So maybe the Zero W's are more of what you might want. If you dont need them all to have wireless then you can just get the Zero's instead.
- charwalker 7y ago
  I'd want a zero for projects. I have a 3B for messing around with and wish I already had PoE out so I could just set and forget a Pihole box or something. At the moment my Pihole instance is on an x86 Raspbian VM for simplicity but I think tonight or this weekend I'll set up the 3B do be a backup and point my router to it as a secondary DNs server. At least it'd be using the Pi vs it sitting in my drawer.
- djaque 7y ago
  By sourcing, I wasn't arguing about their price. The problem is that it's hard to find anyone who will sell you more than a handful of them at low prices. I haven't checked recently, but at least that's what I remember.
mappu 7y ago
> most of the applications I want to use are x86 only Really? What software in this space (headless server cluster) is still x86 only?
- djaque 7y ago
  Unfortunately some proprietary scientific software is compiled for x86 only and we just get access to the binaries. I don't like it, but that's just how some fields work.
TwoNineFive 7y ago
This is purely for education and show. I worked on two top-50 supercomputer clusters for science applications. You learn a lot by doing. You find out what happens to your electrical power, TFTP, DHCP, and syslog servers, and other infrastructure when those hundreds/thousands of nodes boot up all at once. Your Cisco switch reloads when you add a secondary IP address to all of your network interfaces because it's CAM table overflows. You learn how to automate without stuff like Ansible and Chef because they don't exist yet.
- intpx 7y ago
  Ha, it was my experience with HPC that lead me to convince our CCIE network admin that CAM tables even could overload on a vlan of that size (we were using layer 3 switching to route tons of other vlans and this particular vlan was very active in traversing those boundaries and was also using multicast and lots of other weird things). We were seeing an issue when where computers on the same switch would lose their ability to see each other, even if they could see other machines on a different switch and everything about it smelled like the cam table was not accepting new entries. I finally convinced him to clear it out on one affected port and it worked.
imtringued 7y ago
That's a pretty stupid reason. The real reason ist that most software doesn't run on a cluster at all and an 8 core Ryzen costs less than $170 and $340 for a full system without a GPU but is significantly faster than a 10 node rasperry pi cluster ( doesnt even include sd card power supply network gear)

by djaque 7y ago

jacquesm 7y ago

The real selling point for me would be physical isolation. If you run all of them single-core you wouldn't have to worry at all about snooping using the various side channel attacks.

Skunkleton 7y ago
Yes, but how many RPis do you need to match the workload of a single x86 core? How much does it cost to cluster and maintain these systems? IMO you would probably have a better story all around if you based your cluster on low end AMD chips, or perhaps even old refurbished systems. That's not to say this isn't cool engineering for engineering's sake. Just that if you are considering practical applications for this, maybe think twice.
- jacquesm 7y ago
  When you need that guarantee of isolation you are picking your solution for different reasons than when you need a particular workload. So there is no point comparing the workload if that wasn't the deciding factor in the first place.
  - Skunkleton 7y ago
    My point is that you can probably get physical isolation for less money on x86.
    - jacquesm 7y ago
      You're going to have to explain a bit more on how you intend to achieve that economy. I'm not aware of a single SOC+peripherals based on x86 that would beat the Pi in price/performance, is there such a thing?

willis936 7y ago

In case people reading play with raspis and aren’t aware: you can reduce power consumption by several percent by disabling peripherals. This wiki has some useful information for those looking to make raspi clusters. https://www.pidramble.com/wiki/benchmarks/power-consumption https://www.pidramble.com/wiki/benchmarks/power-consumption

giancarlostoro 7y ago

The Raspberry Pi Zero's are ridiculously affordable. For the cost of a regular Pi 3 you can get one thats wireless with a case with 3 covers for the case (1 for the Raspberry Pi camera). So maybe the Zero W's are more of what you might want. If you dont need them all to have wireless then you can just get the Zero's instead.

charwalker 7y ago
I'd want a zero for projects. I have a 3B for messing around with and wish I already had PoE out so I could just set and forget a Pihole box or something. At the moment my Pihole instance is on an x86 Raspbian VM for simplicity but I think tonight or this weekend I'll set up the 3B do be a backup and point my router to it as a secondary DNs server. At least it'd be using the Pi vs it sitting in my drawer.
djaque 7y ago
By sourcing, I wasn't arguing about their price. The problem is that it's hard to find anyone who will sell you more than a handful of them at low prices. I haven't checked recently, but at least that's what I remember.

mappu 7y ago

> most of the applications I want to use are x86 only Really? What software in this space (headless server cluster) is still x86 only?

djaque 7y ago
Unfortunately some proprietary scientific software is compiled for x86 only and we just get access to the binaries. I don't like it, but that's just how some fields work.

TwoNineFive 7y ago

This is purely for education and show. I worked on two top-50 supercomputer clusters for science applications. You learn a lot by doing. You find out what happens to your electrical power, TFTP, DHCP, and syslog servers, and other infrastructure when those hundreds/thousands of nodes boot up all at once. Your Cisco switch reloads when you add a secondary IP address to all of your network interfaces because it's CAM table overflows. You learn how to automate without stuff like Ansible and Chef because they don't exist yet.

intpx 7y ago
Ha, it was my experience with HPC that lead me to convince our CCIE network admin that CAM tables even could overload on a vlan of that size (we were using layer 3 switching to route tons of other vlans and this particular vlan was very active in traversing those boundaries and was also using multicast and lots of other weird things). We were seeing an issue when where computers on the same switch would lose their ability to see each other, even if they could see other machines on a different switch and everything about it smelled like the cam table was not accepting new entries. I finally convinced him to clear it out on one affected port and it worked.

imtringued 7y ago

That's a pretty stupid reason. The real reason ist that most software doesn't run on a cluster at all and an 8 core Ryzen costs less than $170 and $340 for a full system without a GPU but is significantly faster than a 10 node rasperry pi cluster ( doesnt even include sd card power supply network gear)