6 ms·
A low power 1U Raspberry Pi cluster server for inexpensive colocation (2021)
- dnemmers 1y agoMaybe it’s just me, but I can’t imagine this thing staying together in one piece after being shipped. Too much double-sided tape and kludging. Could work well at home, however.
- dnemmers 1y agoPrevious comments: https://news.ycombinator.com/item?id=27862967 https://news.ycombinator.com/item?id=27862967
- asteroidburger 1y agoOnly 16GB of DDR4 and 1.2TB of storage is not exactly a lot, especially when it’s spread across all of those nodes.
- tgv 1y agoWhat do you expect to run on them then? I run 3 user facing services plus their test environments plus the database on the same 8GB server, and half of the memory is free. The database takes about 10GB (growing slowly). There's also a 10GB media directory that grows slowly. Most of it is images, I think, and videos are short. But it'll be quite a while before reaching 1TB.
- crinkly 1y agoI’ve found colo companies to be pretty fussy about what hardware they will accept. Ours would reject this. Edit: and problem with micro clusters like this is always the IPv4 costs.
- petesergeant 1y agoI was going to ask, if colos providers will allow a hacked together chassis like this, what’s to stop someone sending them an intentionally or unintentionally destructive device?
- arnon 1y agoI like the idea but this is not going to last long Good start though!
- postpawl 1y agoProject author here. This project is 4 years old at this point, and now it probably makes more sense to use Mac minis or mini pcs. I also wouldn’t rely on cheap colocation for anything security sensitive or critical. They gave my same block of IPs to another customer at some point and there were issues with IP conflicts (eventually got resolved). It lasted for about 3 years and the colocation company went bankrupt and got bought by another company, so they returned the hardware. I’m surprised a technical failure didn’t kill it.
- tonyhart7 1y agoYeah with 30 bucks a month for co loc, I cant expect them to run for years even if they can sustain that, how the heat and energy health for that cheap building
- noosphr 1y agoDo you know what the most cost effective hardware for general internet stuff is currently? I do ML and have to deal with removing tens of kilowatts from a small closet when it comes to on prem stuff - mainly for compliance reasons - and I have no idea what good, cheap low power hardware for web, sql and similar servers is.
- reactordev 1y agoFor general internet stuff, a Mac mini or any SFF pc will be just fine. For ML, you’ll need at least a dozen or more thousand for GPU’s if you run your own inference. If you use a 3rd party, like OpenAI, it’s just an API call and you can do that on your SFF mini or pi. Web hosts can start at $10 (or free + internet) and GPU hosts can start at $4,000 USD. At peak, a “cluster node” could be $10,000 and a GPU node could be $80,000. The question you have to ask yourself is: what are your requirements.
- p0w3n3d 1y agoMac minis are quite expensive. I'm trying to rebuild my home server and it will be probably Chinese minipc Ryzen 5700u, still tdwp 25 IIRC
- hawk_ 1y agoWhat is the use of colo with an arbitrary provider? (Asking as I have only heard of colo in the context of say an exchange or something else specific)
- Catbert59 1y agoIn 2025 I'd go for an Intel n100/n150/n305 with 32GB RAM (in officially supported). If nothing rotates nothing can break. At work we have ~10 of these passive cooled TopTon n100 with their 5x Intel i226-IV 2.5GE interfaces laying around for emergency router setups. They are great for a lot of things. But be careful: starting with the n150 you will need active cooling.
- rented_mule 1y ago100%!!! I have an n305 with the CPU thermally bonded to its small aluminum case with a quiet 80 mm Noctua fan screwed into the fins of the case. The manufacturer on Ali Express said the fan is optional, but it can get to ~85°F in the room where the computer is, so I want to be careful. At idle, the CPU reports 5-10°F above the room's temperature. It has 10 TB spread across 3 SSDs and 2 x 10 TB spinning drives attached. It's a Time Machine target for a handful of Macs and a Borg Backup target for several machines, including some across the internet using Tailscale. It's also running Home Assistant with AppDaemon (with dozens of little apps), Frigate (object detection for 3 Ethernet-connected cameras using a Google Coral TPU over USB), Paperless-ngx (15 GB of PDFs), LibrePhotos (1.2 TB of photos), Syncthing, Tiny Tiny RSS, a UniFi Controller, distinct PostgreSQL instances for various of those, and more. I count 21 Docker containers running right now, and not everything is containerized. The spinning drives are powered down with a smart plug for all but 1-2 hours at night for backups. With those off, the thing sips power... 10-15 W most of the time with occasional spikes up to ~30 W when LibrePhotos is doing facial recognition or Paperless-ngx is updating its ML models. It never feels slow. I've been running one or more servers at home for 30+ years, and this single machine handles everything so much better than any combination of machines I've had.
- dwood_dev 1y ago32GB is official, but 48GB works just fine. 64GB SODIMMs are now available and there are multiple reports of them working fine with the N305[0]. It is highly likely that it will work fine with the N100 as well. 0: https://www.reddit.com/r/homelab/comments/1m8fgec/intel_n305_running_stable_with_64gb_of_ram/ https://www.reddit.com/r/homelab/comments/1m8fgec/intel_n305...
- bullen 1y ago
- mvip 1y agoIf you like this stuff, you should check out what the guys at Mythic Beasts are doing. They’re squeezing a ton of Raspberry Pi’s into racks. They also host the Raspberry Pi website on pi’s. We used them for a while and there are some photos here https://www.screenly.io/blog/2023/05/25/updated-qc-rig/ https://www.screenly.io/blog/2023/05/25/updated-qc-rig/
- rahimnathwani 1y agoThe first hosted server I ever rented (~20 years ago) was as the first customer of Black Cat Networks (which was later acquired by Mythic Beasts). It was a small Apple machine running Debian. ISTR it was an Apple TV (1st gen), but it might have been a Mac Mini.
- procaryote 1y agoPretty fun! Pi's are powerable from the header pins, so you could save the usb adapters and route power directly from the relays to the power pins. I'd also be tempted to add a way to access the serial consoles and the power button-equivalent pins of the pis for a lom-equivalent. It might be doable with a pico or two. Nowadays with pi5s you could also of course hook a m2 board up to the pci lane and skip the m2 enclosures. I'd personally prefer a recessed push-button power switch too; the switch you use would make me nervous something would drop on it and turn the system off
- mmastrac 1y agoI've done this in the past and it works quite well. I recommend that you buy a 5V, 20A power supply, crank it up to 5.3V-ish, and power directly to each Pi's header through the relay. Put a big cap over the Pi's power pins to handle transient power blips which was the most likely cause of most of my Pi lockups over time. The nice thing about relay power is that you don't need a power button. In my case I actually had a little arduino running a USB stack that could toggle GPIOs to power cycle the Pis if they wedged: https://github.com/mmastrac/pi-power-vusb https://github.com/mmastrac/pi-power-vusb (I forgot that I even added default power states and power-on sequencing there) Serial is definitely a nice to have. One USB-serial per pi, with one overall controller Pi that can aggregate it all. I would add that TFTP boot for each Pi is also really convenient. This is pretty easy to set up. Dedicate one Pi to manage the cluster power, serial and TFTP and you have a pretty robust setup.
- bullen 1y agoI use 3.5mm barreljack splitter cables and 3.5mm barrel to USB micro/C adapters.
- geniusalsdja 1y ago[dead]
- pluto_modadic 1y agoCould be fun for an ipv6 cluster
- xvfLJfx9 1y agoI'd recommend just getting a piblade and mounting them...
- deleted 1y ago[deleted]
- bullen 1y agoI built the cluster thick instead: https://github.com/tinspin/rupy https://github.com/tinspin/rupy And selfhost on home fiber. Saves space and cools silently. Mixing old 2 and 4 for different use cases. Raspberry 5 and 3588 are too hot. Not in the picture Mean Well 50W passive PSU.
- omarqureshi 1y agoSimilar setup (not in a 1U). I have 3 Pi 5's in my AV rack with M2 and PoE Hats - they actually work fairly well and bonus? Don't need to power them. I could get a 1U enclosure but, it's fine. The only issue is one of the PoE Hats fan is catching something (though nothing i can see), so on occasion it will need persuasion to be quiet
- rbanffy 1y agoThere are many cluster boards that allow plugging compute module boards that have an onboard switch. Such an arrangement would provide a much denser system. Making a new one, however, requires a lot of work. I'm not even sure how you do ethernet over PCB traces. One project I keep telling myself I'll eventually do is to make a cluster board with 32 Octavo SoMs (each with 2 ethernets, CPU, GPU, RAM, and some flash), and a network switch (or two). And 32 activity LEDs on the side so a set of 16 boards will look like a Connnection Machine module.
- nateb2022 1y ago> This repo is about designing a server that fits within the 1U space and 1A @ 120v power constraint while maximizing computing power, storage, and value. For a $800 setup and $30-50/mo in colocation fees, I think this is a lot worse value overall than a lot of hosting providers, e.g. Hetzner provides servers that start at ~$36/mo with around 64GB RAM and 1TB storage. Even if someone absolutely wanted to configure + provide their own hardware to colocate, you could probably put together a much nicer server for less by scavenging parts off of eBay. Fun yes, but practical no.