3 ms·
Would it be feasible to build a whole cluster of these for compute power?
by jb1991 5y ago
Would it be feasible to build a whole cluster of these for compute power?
- ignoramous 5y agoThere are companies being built around raspberrypi-in-a-server-farm, for ex balena.io
- robbedpeter 5y agoThe limited data transfer rate makes clustering ineffective, unfortunately.
- qbasic_forever 5y agoNo, you need a lot of adapters to add ethernet, durable mass storage, etc. over USB OTG. Since these are just Raspberry Pi 3 SOCs effectively you could pick some of those up for the same price as this new board and all the USB dongles and adapters you'd need.
- geerlingguy 5y agoMaximum network bandwidth is ~220 Mbps, and that's with a massive USB Ethernet dongle hanging off each Zero 2. WiFi is a pretty anemic ~35 Mbps. So unless your clustering needs are very low-performance and don't need much in the way of networked bandwidth, I would stay away from trying to cluster Zeros. The Zero 2 is slightly less-bad than the original Zero W (I actually built a tiny K3s cluster with it way back when...), but still not a great idea. It's extremely slow and the 512 MB of RAM limits what you can do a lot.
- lrem 5y agoMost of those tinkerer boards spend too much power on low-bandwidth connectivity and use too old cpus to ever be practical for compute. Outright wrong orders of magnitude.
- jabl 5y agoFor learning about clustering, sure, go wild. Or maybe use the latest and greatest Pi (is that Pi4 ATM?) to avoid issues with extremely low end stuff (e.g. even disregarding applications, can you run a "cluster stack" on this without OOM etc.?). For "real work", nope, not worth it. For that, look into what e.g. the giant IAAS providers (AWS, Azure, GCP) or the top500 list are running on.
- jb1991 5y ago> look into what e.g. the giant IAAS providers How do you find that out?
- jabl 5y agoWell, occasionally they give presentations describing their hardware infrastructure. And you can deduce something from their documentation about available VM types. Anyway, the idea wasn't to give an exact recipe but rather to give a glimpse on what is generally considered a cost-effective compute platform.
- chokma 5y agoI build a Hadoop cluster with 5 Raspberry Pi-2B in 2015. It works, but the ratio of power consumption to compute performance was worse compared to using a standard PC. edit: with the huge increase in performance, Pi-4 models may now be competitive.
- behohippy 5y agoThat's awesome, I did the same project when I was working for Hortonworks. I had 8 nodes and it was really limited by memory. I got a few MR jobs running and some benchmarks, but not a great cluster overall.