6 ms·

What could you do with a similar setup? I mean: what's the purpose and/or the advantages compared to a "classical" solution ?

canofbars 6y ago
Its cool and fun and teaches you a skill at a low price point. There is essentially no functional reason to do this over a single high power server unless for some reason you need a bunch of ARM servers for something.
- OJFord 6y ago
  All fourteen Pis, even v4s (which are significantly hungrier), would be lower power (as in electrical draw) than even a feeble (as in compute power) traditional machine. If you can distribute your workload (or it's not intensive anyway), i.e. having no single machine with a faster processor, then I think it's a good choice, cheaper to run. Edit: I'm happy to be corrected if down-voters would like to reply... A Pi 4 draws 6-7W at full load. A former desktop PC turned server likely idles over 100W. They're different beasts, so I'm not saying 'a Pi is better than a PC', I'm saying 'if your load could also be handled by a (bunch of) Pi(s), then all else being equal that might be better than the PC'. Of course all else is never equal, ymmv, you may already have the old PC, you may need very many Pis, so even if pennies cheaper to run it might take years and years to break even on the Pi acquisition cost.
  - scns 6y ago
    Did not downvote but desktops idle way below 100w.
    - OJFord 6y ago
      Idle as in powered on, not sleeping, just 'no' workload? I stand corrected if so, I didn't realise that was the case. I also like the idea of having an easy homogeneous setup, each Pi having one disk, vs. my alternative (old PC parts) being something like '4 SATA ports on this one, and 6 on that one, but one's broken, less one on each for the boot disk' - I quite like the idea of having a bunch of Pis, and one image that I apply to all of them, which can include details like /dev/sda being its only data disk. That's not really a functional advantage, just appealing for keeping it simple to come back to and remember/understand how I set it up when I need to tweak something (and really easy to add another one increasing available cpu/ram/disk).
      - hopw_roewur_ne 6y ago
        This high end system draws ~35W idle: Intel Core i9-10900K @ 5,0 GHz, AMD Radeon RX 6900 XT (16 GB), Asus ROG Maximus XII Hero Wi-Fi, 2× 16 GB G Skill Trident Z Royal 3 600 MHz 16-16-16-36, Samsung 970 Evo M.2 1 TB, Samsung 860 Evo 1 TB, Seasonic Prime Ultra Titanium, 1 000 W. Picture in Swedish, from graphics card review comparing average power draw for the entire system for various GPUs, idle and while benchmarking in Metro Exodus, numbers rounded to nearest multiple of 5: https://cdn.sweclockers.com/artikel/diagram/22452?key=2c6f21ab413bb65ebb16423ffaa68db6 https://cdn.sweclockers.com/artikel/diagram/22452?key=2c6f21... If the direct picture link doesn't work, it's under "Effektmätning" here: https://www.sweclockers.com/test/30908-amd-radeon-rx-6900-xt/9 https://www.sweclockers.com/test/30908-amd-radeon-rx-6900-xt...
    - fortran77 6y ago
      Exactly. Take a look at the "ENERGY STAR ® Power and Performance Data Sheet" for Dell "office" desktops. Many are around 10 watts for idle.
  - detaro 6y ago
    It's not that difficult to have a reasonably modern desktop PC idling more in the 10-20W range. For 100W you need to do something seriously wrong, or add extras that would blow up the Pi usage too (e.g. a home server full of hard disks will of course use more power, but so would Pis with the same number of drives)
  - deleted 6y ago
    [deleted]
  - nebalee 6y ago
    Don't forget that Pis also draw quite some power while idle. The Pi 4 draws around 3.4 Watts [1], or around 2.7 Watts since they are running headless in a server setup you'd probably disable HDMI [2]. Times fourteen that's 37.8 Watts just sitting there. An idling headless Desktop PC will be below that for sure. [1] https://magpi.raspberrypi.org/articles/raspberry-pi-specs-benchmarks https://magpi.raspberrypi.org/articles/raspberry-pi-specs-be... [2] https://www.pidramble.com/wiki/benchmarks/power-consumption https://www.pidramble.com/wiki/benchmarks/power-consumption
- dvfjsdhgfv 6y ago
  A good functional reason is when you need to have high redundancy for low-intensity jobs while keeping power consumption ridiculously low. As long as you don't use SD cards, Pis can run for years without stability issues.
- gtvwill 6y ago
  lol? what if you don't need a heap of compute power but require a lot of redundancy? A single server is not good for that...Also not every site has internet so yes having small redundant easily replaceable nodes on-site does have plenty of use cases. Y'all negative folks against this seem to have very restricted use cases in mind. Not all redundant rack servers live in a 36U case in a air-conditioned internet connected server farm. There are plenty of use cases outside of this.
contingencies 6y ago
Purpose: Learn linux clustering/diskless nodes. Advantages: Cheaper than conventional servers, limited physical form factor suitable for constrained environments such as apartments, luggage, etc. Low power consumption, suitable for mobile/off-grid/renewable. Recommended software: https://clusterlabs.org https://clusterlabs.org https://www.kernel.org/doc/html/latest/admin-guide/nfs/nfsroot.html https://www.kernel.org/doc/html/latest/admin-guide/nfs/nfsro... https://wiki.gentoo.org/wiki/Diskless_nodes https://wiki.gentoo.org/wiki/Diskless_nodes Commercial advantage: Run highly cost and power efficient data centers, call centers, etc. Eliminate redundant state. Remove manual configuration rollout. Infrastructure as code. Full CI/CD with kernel-level control, per node, for any OS. Basically becoming a lost art in this day of "outsource to cloud". With SDN knowledge, confidently build your own cloud provider.
- skohan 6y ago
  Is the power consumption really that good? I mean comparing to just getting a single 5950x or something would this actually win in terms of performance per watt?
- z3t4 6y ago
  You could rent a couple of 1USD/month VPS, or if really on a budget rent a bunch of containers with ipv6 for just a few cents
  - contingencies 6y ago
    You absolutely could. But you wouldn't learn how to build a system that recovers from yanking a power or network cable, wouldn't own it, and wouldn't be able to run it on your terms. You'd need a credit card with an American-run payments network to operate, with a declared identity and a bank account, for which a prerequisite is a few months of stable living and income in a particular jurisdiction, itself something large portions of the world do not have access to. Further you'd need to be happy with the fact none of your data was in your country, under your physical control, and never would be. But feel free to ignore the rest of the world and the fact that all this rental crap runs on open source software, just throw in the towel and be a rent-payer: a strategy that has its perks and efficiencies, for sure. But don't discourage those who seek to learn more.
elitepleb 6y ago
Clustering SBC's is a great learning experience, and makes for a great budget friendlier home lab setup.
pachico 6y ago
I have a friend that uses similar setups to tech kubernetess but previously he also used it as example for all sorts of distributed computing (when Hadoop was still a cool thing to do)
Tepix 6y ago
If you need a cheap dedicated (for privacy) server to run a mail server and perhaps a private cloud with Nextcloud, this is ideal.
- ekianjo 6y ago
  Nextcloud would be much cheaper to run on a x86 board.
  - kingosticks 6y ago
    Can you expand on why?
    - deleted 6y ago
      [deleted]
    - sirn 6y ago
      My guess would be Nextcloud put a lot of load on both CPU and disk (for thumbnail generation, file encryption, etc.) making Raspberry Pi which lack performance in both front not very suitable for the task. I tried running Nextcloud on Raspberry Pi 4 (4GB) a while ago, and hitting performance issue on SD card very fast (initial DB import take very long). I tried to solve this by moving all Nextcloud data into iSCSI disk but RAM & CPU became a problem once I started adding more files. After a week or so, I tore everything down and went back to a VPS instance. It might works better with Raspberry Pi cluster on Kubernetes, though. (This very Raspberry Pi is now happily running as a DVB streaming server.)
      - kingosticks 6y ago
        Thanks. I keep meaning to try it out but booting from an SSD instead. I was hoping anything intensive (e.g. thumbnail generation) would be a one-time thing so not noticeable in daily usage. Perhaps I will re-think, I have a ThinkCentre M75q I could use but that seems overkill.
      - ekianjo 6y ago
eli 6y ago
It’s fun to hack together. Not everything needs a clear business case with strong ROI.
- drannex 6y ago
  This. I think the Hacker News mindset has substantially shifted in the last few years from the "Hacker" mentality of doing things because they are fun and knowledgeable, and more towards Enterprise News.. Not everything needs a business case, some things (most things) should be done just because you want to see if you can. If someone really needs to think in items of ROI and that everything needs a business case, then this ideology of doing things for fun and testing limits is the only way to really innovate.
rwmj 6y ago
I bought a Turing Pi 1 . It's a 7 node mini-cluster, not very expensive (I think about £300 fully populated), that fits into an MiniITX-style case with a single small power supply and single ethernet exposed to the world. You can easily pick the thing up with one hand. It's certainly not powerful, each node is a very modest ARM processor with "only" a gigabyte of RAM and 8 GB of local storage. The alternatives are much more powerful, but far more expensive and not very attractive in other ways. Blade server? Huge, noisy, power-hungry and I wouldn't get much change from many thousands of pounds. Rack of NUCs? Powerful of course but far more expensive, large, separate power bricks, ethernet switch and lots of wiring required. VMs running on a desktop? Still more expensive, larger, and not the same tangible experience as having a real cluster. VMs on a laptop? Laptops at the same price point would have too little RAM to run lots of VMs well. So these Raspberry Pi clusters fit a niche which is not well served by other solutions.
fulafel 6y ago
You can neatly mount your Pis in your rack where you have the rest of your gear supported by network switches, power delivery, cabling, cooling, good internet connectivity connectivity (if housed in a machine room or colo) etc.
sorisos 6y ago
Could be used for testing hardware+firmware. Example connect 14 hardware revisions of a $SOME_HARDWARE to each RPi and test firmware changes.

canofbars 6y ago

Its cool and fun and teaches you a skill at a low price point. There is essentially no functional reason to do this over a single high power server unless for some reason you need a bunch of ARM servers for something.

OJFord 6y ago
All fourteen Pis, even v4s (which are significantly hungrier), would be lower power (as in electrical draw) than even a feeble (as in compute power) traditional machine. If you can distribute your workload (or it's not intensive anyway), i.e. having no single machine with a faster processor, then I think it's a good choice, cheaper to run. Edit: I'm happy to be corrected if down-voters would like to reply... A Pi 4 draws 6-7W at full load. A former desktop PC turned server likely idles over 100W. They're different beasts, so I'm not saying 'a Pi is better than a PC', I'm saying 'if your load could also be handled by a (bunch of) Pi(s), then all else being equal that might be better than the PC'. Of course all else is never equal, ymmv, you may already have the old PC, you may need very many Pis, so even if pennies cheaper to run it might take years and years to break even on the Pi acquisition cost.
- scns 6y ago
  Did not downvote but desktops idle way below 100w.
  - OJFord 6y ago
    Idle as in powered on, not sleeping, just 'no' workload? I stand corrected if so, I didn't realise that was the case. I also like the idea of having an easy homogeneous setup, each Pi having one disk, vs. my alternative (old PC parts) being something like '4 SATA ports on this one, and 6 on that one, but one's broken, less one on each for the boot disk' - I quite like the idea of having a bunch of Pis, and one image that I apply to all of them, which can include details like /dev/sda being its only data disk. That's not really a functional advantage, just appealing for keeping it simple to come back to and remember/understand how I set it up when I need to tweak something (and really easy to add another one increasing available cpu/ram/disk).
    - hopw_roewur_ne 6y ago
      This high end system draws ~35W idle: Intel Core i9-10900K @ 5,0 GHz, AMD Radeon RX 6900 XT (16 GB), Asus ROG Maximus XII Hero Wi-Fi, 2× 16 GB G Skill Trident Z Royal 3 600 MHz 16-16-16-36, Samsung 970 Evo M.2 1 TB, Samsung 860 Evo 1 TB, Seasonic Prime Ultra Titanium, 1 000 W. Picture in Swedish, from graphics card review comparing average power draw for the entire system for various GPUs, idle and while benchmarking in Metro Exodus, numbers rounded to nearest multiple of 5: https://cdn.sweclockers.com/artikel/diagram/22452?key=2c6f21ab413bb65ebb16423ffaa68db6 https://cdn.sweclockers.com/artikel/diagram/22452?key=2c6f21... If the direct picture link doesn't work, it's under "Effektmätning" here: https://www.sweclockers.com/test/30908-amd-radeon-rx-6900-xt/9 https://www.sweclockers.com/test/30908-amd-radeon-rx-6900-xt...
  - fortran77 6y ago
    Exactly. Take a look at the "ENERGY STAR ® Power and Performance Data Sheet" for Dell "office" desktops. Many are around 10 watts for idle.
- detaro 6y ago
  It's not that difficult to have a reasonably modern desktop PC idling more in the 10-20W range. For 100W you need to do something seriously wrong, or add extras that would blow up the Pi usage too (e.g. a home server full of hard disks will of course use more power, but so would Pis with the same number of drives)
- deleted 6y ago
  [deleted]
- nebalee 6y ago
  Don't forget that Pis also draw quite some power while idle. The Pi 4 draws around 3.4 Watts [1], or around 2.7 Watts since they are running headless in a server setup you'd probably disable HDMI [2]. Times fourteen that's 37.8 Watts just sitting there. An idling headless Desktop PC will be below that for sure. [1] https://magpi.raspberrypi.org/articles/raspberry-pi-specs-benchmarks https://magpi.raspberrypi.org/articles/raspberry-pi-specs-be... [2] https://www.pidramble.com/wiki/benchmarks/power-consumption https://www.pidramble.com/wiki/benchmarks/power-consumption
dvfjsdhgfv 6y ago
A good functional reason is when you need to have high redundancy for low-intensity jobs while keeping power consumption ridiculously low. As long as you don't use SD cards, Pis can run for years without stability issues.
gtvwill 6y ago
lol? what if you don't need a heap of compute power but require a lot of redundancy? A single server is not good for that...Also not every site has internet so yes having small redundant easily replaceable nodes on-site does have plenty of use cases. Y'all negative folks against this seem to have very restricted use cases in mind. Not all redundant rack servers live in a 36U case in a air-conditioned internet connected server farm. There are plenty of use cases outside of this.

contingencies 6y ago

Purpose: Learn linux clustering/diskless nodes. Advantages: Cheaper than conventional servers, limited physical form factor suitable for constrained environments such as apartments, luggage, etc. Low power consumption, suitable for mobile/off-grid/renewable. Recommended software: https://clusterlabs.org https://clusterlabs.org https://www.kernel.org/doc/html/latest/admin-guide/nfs/nfsroot.html https://www.kernel.org/doc/html/latest/admin-guide/nfs/nfsro... https://wiki.gentoo.org/wiki/Diskless_nodes https://wiki.gentoo.org/wiki/Diskless_nodes Commercial advantage: Run highly cost and power efficient data centers, call centers, etc. Eliminate redundant state. Remove manual configuration rollout. Infrastructure as code. Full CI/CD with kernel-level control, per node, for any OS. Basically becoming a lost art in this day of "outsource to cloud". With SDN knowledge, confidently build your own cloud provider.

skohan 6y ago
Is the power consumption really that good? I mean comparing to just getting a single 5950x or something would this actually win in terms of performance per watt?
z3t4 6y ago
You could rent a couple of 1USD/month VPS, or if really on a budget rent a bunch of containers with ipv6 for just a few cents
- contingencies 6y ago
  You absolutely could. But you wouldn't learn how to build a system that recovers from yanking a power or network cable, wouldn't own it, and wouldn't be able to run it on your terms. You'd need a credit card with an American-run payments network to operate, with a declared identity and a bank account, for which a prerequisite is a few months of stable living and income in a particular jurisdiction, itself something large portions of the world do not have access to. Further you'd need to be happy with the fact none of your data was in your country, under your physical control, and never would be. But feel free to ignore the rest of the world and the fact that all this rental crap runs on open source software, just throw in the towel and be a rent-payer: a strategy that has its perks and efficiencies, for sure. But don't discourage those who seek to learn more.

elitepleb 6y ago

Clustering SBC's is a great learning experience, and makes for a great budget friendlier home lab setup.

pachico 6y ago

I have a friend that uses similar setups to tech kubernetess but previously he also used it as example for all sorts of distributed computing (when Hadoop was still a cool thing to do)

Tepix 6y ago

If you need a cheap dedicated (for privacy) server to run a mail server and perhaps a private cloud with Nextcloud, this is ideal.

ekianjo 6y ago
Nextcloud would be much cheaper to run on a x86 board.
- kingosticks 6y ago
  Can you expand on why?
  - deleted 6y ago
    [deleted]
  - sirn 6y ago
    My guess would be Nextcloud put a lot of load on both CPU and disk (for thumbnail generation, file encryption, etc.) making Raspberry Pi which lack performance in both front not very suitable for the task. I tried running Nextcloud on Raspberry Pi 4 (4GB) a while ago, and hitting performance issue on SD card very fast (initial DB import take very long). I tried to solve this by moving all Nextcloud data into iSCSI disk but RAM & CPU became a problem once I started adding more files. After a week or so, I tore everything down and went back to a VPS instance. It might works better with Raspberry Pi cluster on Kubernetes, though. (This very Raspberry Pi is now happily running as a DVB streaming server.)
    - kingosticks 6y ago
      Thanks. I keep meaning to try it out but booting from an SSD instead. I was hoping anything intensive (e.g. thumbnail generation) would be a one-time thing so not noticeable in daily usage. Perhaps I will re-think, I have a ThinkCentre M75q I could use but that seems overkill.
    - ekianjo 6y ago

eli 6y ago

It’s fun to hack together. Not everything needs a clear business case with strong ROI.

drannex 6y ago
This. I think the Hacker News mindset has substantially shifted in the last few years from the "Hacker" mentality of doing things because they are fun and knowledgeable, and more towards Enterprise News.. Not everything needs a business case, some things (most things) should be done just because you want to see if you can. If someone really needs to think in items of ROI and that everything needs a business case, then this ideology of doing things for fun and testing limits is the only way to really innovate.

rwmj 6y ago

I bought a Turing Pi 1 . It's a 7 node mini-cluster, not very expensive (I think about £300 fully populated), that fits into an MiniITX-style case with a single small power supply and single ethernet exposed to the world. You can easily pick the thing up with one hand. It's certainly not powerful, each node is a very modest ARM processor with "only" a gigabyte of RAM and 8 GB of local storage. The alternatives are much more powerful, but far more expensive and not very attractive in other ways. Blade server? Huge, noisy, power-hungry and I wouldn't get much change from many thousands of pounds. Rack of NUCs? Powerful of course but far more expensive, large, separate power bricks, ethernet switch and lots of wiring required. VMs running on a desktop? Still more expensive, larger, and not the same tangible experience as having a real cluster. VMs on a laptop? Laptops at the same price point would have too little RAM to run lots of VMs well. So these Raspberry Pi clusters fit a niche which is not well served by other solutions.

fulafel 6y ago

You can neatly mount your Pis in your rack where you have the rest of your gear supported by network switches, power delivery, cabling, cooling, good internet connectivity connectivity (if housed in a machine room or colo) etc.

sorisos 6y ago

Could be used for testing hardware+firmware. Example connect 14 hardware revisions of a $SOME_HARDWARE to each RPi and test firmware changes.