7 ms·
Six-Year homelab history in pictures
- sgt 8y agoCool. This puts my homelab to shame. I've basically got an old i5 server running Linux (non SSD), an unstable HP DL160 (never buy HP, kids!) and a Raspberry Pi for home automation. I've also got lots of additional hardware that I rarely power up such as Sun Blade 2000's, SGI's etc.
- blattimwind 8y agoI've been trying to get rid of my PowerPC and SPARC machines forever... it is a very slow going business.
- ajmarsh 8y agoNice write up and photos. I used to maintain a lab like this only my cable management was nowhere near as nice as his. I've replaced this all with a combination of AWS and an Antsle. I miss the hands-on hardware time but don't miss the fan noise.
- JohnJamesRambo 8y agoWhat do you store on the servers and use them for?
- vvanders 8y agoNot my lab but it varies from homelab to homelab. A lot of it is using it as a safe place to understand/experiment with new tech without breaking anything in a production/work environment. Some people also use them to run Plex/Pihole/Pfsense and the like as well.
- awat 8y agoI think this is a pretty good summation. I'm fairly active at the r/homelab subreddit and this is pretty on point. Usually there is some combination of upgraded home media/routing and then on top of that a learning and experimenting section based on your specific goals. For others though it really is a hobby in the sense of building to build.
- moftz 8y agolinux isos
- ajmarsh 8y agoCertification prep mostly, some concepts are best not tested out at work.
- noonespecial 8y agoI use mine simply as a hobby. For a while, I tried to justify it to myself that it was "for work". But it wasn't. I play model datacenter the way my grandfather played model railroad. Good news is that I spend way less than he did. The future is awesome.
- ct0 8y agoWith that amount of storage, the most likely reason for all the gear is hosting media.
- jclay 8y agoI recently set up Pfsense on an old box I had lying around. The practical uses i’ve found are: setting up the PfsenseNG DNS adblocker and setting up an OpenVPN server so I can VPN into my LAN away from home instead of port forwarding. I also just set up Dynamic DNS so that I can do this with a domain. Can anyone share other use cases they’ve found to be practical with their homelab setup?
- tfmatt 8y agoTo expand on the VPN benefits you already have in place... grab a vnc app for Android and VPN in then vnc to a machine from your phone. All of the sudden you are sitting at an Linux desktop on your phone. With DLNA casting you can literally now use your phone as a full blown computer casting to a monitor (assuming you have some sort of peripherals). That was always my holy grail.
- Ajedi32 8y ago- Fine-grained QoS controls via PfSense - Guest Wi-Fi VLAN with UniFi + PfSense - NAS with automatic backups of client machines via Syncthing - Automatic file history of backups with ZFS Snapshots via FreeNAS - Automatic mirroring of backups to the cloud with Duplicati + Sia - Home media server via Plex - Home automation / web dashboard via Home Assistant - Minecraft server with MineOS in a VM Those are all things I either have working, or am currently working on setting up on my own network.
- jclay 8y agoCan you expand on the QoS setup you’re using? I’ve recently read about the fq_codel addition to Pfsense and wondering if anyone here has been using that. I’m also looking for more performance related tunes as well. To that end, I’ve configured Cloudflares DNS but not sure what else yields results. It sounds like squid caching is mostly useless due to most traffic being HTTPS.
- Ajedi32 8y agoIt's actually a pretty simple queue-based setup right now. My primary intent was to ensure that bulk downloads and uploads don't interfere with my latency while gaming. I did look into fq_codel but I don't believe that was a practical option on PfSense back when I was setting everything up. I might have to revisit that now that it has native support in PfSense 2.4.4.
- kasey_junk 8y agoMy problem with homelabs (mine and others) is the power management. Finding hardware that triangulates to energy efficient/good proxy for production/cost effective is really hard to do.
- secabeen 8y agoNoise is the other issue. I found a nice fanless switch, but my server was noisy for years until I got serious about fans and switched everything to 4-pin Noctuas with PWM set at 50%.
- chrisper 8y agoI made my server near silent (it's basically just a desktop computer) but now the biggest noise comes from the disks. I can't stand the noise. Not sure how to make disks quiet though.
- acroback 8y agoGet some SSDs
- WrtCdEvrydy 8y agoPlay with the acoustic settings or switch to NAS-grade drives (WD Red)
- icedchai 8y agoI just ignore it. I have about 10 servers in my basement. Total electric bill is like $160/month.
- ianhowson 8y agoI've got a T710 pulling about 200W continuous and it contributes (.28 * .2 * 24 * 30) = $40/mo to our electricity bill in California. A Hetzner dedi is more cost effective for some workloads.
- bjelkeman-again 8y ago
- jackallis 8y agofor someone who has zero knowledge of networking, looking at those pics - WOOOOW!!
- kazen44 8y agoIf you are actually interested in learning more about networking, i cannot recommend starting your own homelab enough. networking seems to be a hit or miss for people, some people find it very logical and "easy" while others seem convinced it's black magic. networking basics is rather easy, the problem is complexity that gets added when you scale. (although i doubt you would reach that point in a homelab though).
- jgys 8y agoI have slightly more than zero knowledge of networking and would like to remedy this. You seem convinced it’s not black magic, so recommend a book/mooc/resource that’ll prove it. :-P
- DrPhish 8y agoThis will explain the basics of LANs in a fun, but very thorough way: https://www.amazon.com/All-New-Switch-Book-Switching-Technology/dp/0470287152 https://www.amazon.com/All-New-Switch-Book-Switching-Technol... And this one will fill out your knowledge on the routing side, although it is rather dry: https://www.amazon.com/Network-Routing-Algorithms-Architectures-Networking-ebook/dp/B005HRE41W https://www.amazon.com/Network-Routing-Algorithms-Architectu...
- johnflan 8y agoLooks like a damn expensive hobby, one which I have a certain affinity for.
- vvanders 8y agoUsually people buy old servers instead of building supermicro. You can pick up an R610/R710 for $2-300 and even the newer R620/R720 for $3-400. As long as you doing mind the noise + power they're not a bad deal.
- deleted 8y ago[deleted]
- Kadin 8y agoAs others have said, it can be done cheaply. The electricity costs more than the hardware at least for most people's setups that I've seen. (Exception is people using stuff like RPi boards from solar panels; that's almost a sub-hobby in itself.) I run mine in a basement in place of a small utility heater. If I'm going to burn the watts anyway, might as well get some processing cycles out of it.
- astrodust 8y agoBe patient and wait for deals you can snipe. You can get a server that was considered high-end circa 2010 for $50. Some high-end Xeon chips are dirt cheap, like $50 each or less, as parts. Certain models were made by the millions and are still good, but got tossed as more energy efficient versions came to market. It's just not economical to use them at scale.
- cbdumas 8y ago"I then got hold of a 3.2TB Micron 9100 PRO NVMe SSD" How does one "get hold" of a $3k SSD? This is awesome though, you clearly put a lot of work into this.
- Vaslo 8y agoI've always wanted to do something like this. Where would someone go about learning how? Would you need a background or are there other resources you could use to learn?
- chrisper 8y agoWell you probably need to have some basic ideas of computers. Then the rest is just lots of googling and trying out etc. You can also just use VMs for this.
- ranger207 8y agoreddit.com/r/homelab has a decent wiki for the hardware side. For the software side, it's mostly DevOps/sysadmin work. DigitalOcean has extensive tutorials for that kind of stuff.
- slyall 8y agoWorried to see the rackmount hardware close to the carpeted floor with a wall directly behind them. Potential for dust and airflow issues. I've had bits and pieces of a home setup over the years (currently just an HP microserver with disks and VMs) but lots of cost and noise for something I probably won't use a lot of.
- _jcwu 8y agoSince I got fiber gbit up and down I try to relocate as much as possible of my homelab to real data centers (Hetzner) since I can't stand the noise. (I have 10ms ping to my server) Then I use wireguard to connect it to my home network and make it act like a server in the home network. (All the traffic even gets directed through home) I still have the main storage server locally though because otherwise I'd not be able to even watch movies if my internet went down. My dream would be an all SSD storage server being near silet.
- indigo945 8y agoI, on the other hand, use a very old pink netbook I got from a garage sale to host my syncthing backups and as a CalDAV server. Hey, it even comes with a USP!
- ipsum2 8y agoI love this idea. You can get decent laptops (i7, 8gb rm) with broken displays for cheap on EBay/Craigslist, and install Linux. Best way to set up a homelab on the cheap.
- WrtCdEvrydy 8y agoThis was actually one of the most cost efficient ways to do scaled low power datacenters for the longest time. Set the fans to max and do direct DC off of a DC UPS.
- mirimir 8y agoThese days, I mainly use three old i5 boxes with 8GB RAM and SSD RAID10. Two with 4x128GB and one with 6x256GB. They're all VirtualBox hosts, and run various pfSense VPN-gateway VMs, Whonix VMs, and other workspace VMs. I also have some old rack-mount servers. But their fan-control systems are all Windows-based, and they're impossibly loud with Linux. I keep meaning to do some crude electrical hack -- basically a manual speed control -- but just haven't gotten around to it. And there's also the fact that I don't need that much muscle locally. Because I like to keep my network activity minimal. Everything is on a 0.25kWh UPS (3kW maximum load). With a kill switch on my desk. The UPS and a pfSense perimeter router/firewall (and a rack with the old servers) live in a well-ventilated closet.
- znpy 8y agoThis (wo|)man is living the dream.
- jgys 8y ago“Person” is way more inclusive and way less clunky.
- chooseaname 8y agoThat's an amazing setup. My little TS140 with it's Xeon running Proxmox does just about all I need. My Synology does the backups and my EdgeRouter-X does the routing to round things out.
- jtl999 8y agoI just have a Skylake Xeon E3 quad core from 2016 as my main sever and an i3-6100 with 8GB of RAM and Intel NICs as my router, but I want more goodies :D
- whalesalad 8y agoThis is very awesome. For those interested, the Reddit.com/r/homelab section is super great for this kind of stuff. I just got a Dell R720 off Craigslist to do an experiment with the Firecracker MicroVM of all things. It's got 48gb of ram and dual hex-core Xeons. It's a lot of fun to play with this kind of gear. Ikea sells a 'rack' (not the Lackrack, https://wiki.eth0.nl/index.php/LackRack https://wiki.eth0.nl/index.php/LackRack) that is really great for holding gear: https://www.ikea.com/us/en/catalog/products/S39031411/ https://www.ikea.com/us/en/catalog/products/S39031411/ Some pix of my recent setup: https://imgur.com/a/xXc6720 https://imgur.com/a/xXc6720 Back in ~2014 I had a similar tiny setup in my basement with a UPS and an older (and much louder) Poweredge 2950: http://i.imgur.com/zxYxuLq.jpg http://i.imgur.com/zxYxuLq.jpg
- jcims 8y agoThe Dell C62x0 series high density compute systems are pretty cool too. Up to 4 nodes per chassis, dual cpu 48 gb per node. Biggest issue is fan noise. You have to tuck them away somewhere. e.g. https://www.ebay.com/itm/Dell-PowerEdge-C6220-GEN-II-4-x-Node-Server-8-x-SIX-CORE-2-60GHz-E5-2630V2-128GB/132907453614?hash=item1ef1e6ccae:g:q6AAAOSwzd9cLj5B https://www.ebay.com/itm/Dell-PowerEdge-C6220-GEN-II-4-x-Nod...
- code4tee 8y ago+1 for using old equipment for home labs. There’s lots of perfectly good hardware out there for cheap that may be beyond it’s useful life for top end production workloads but is still great for other projects or just tinkering.
- walrus01 8y agoAnyone who is thinking of doing this should consider how loud and annoying 1RU sized servers and switches are. They're not designed to be used in the same room as people for any extended amount of time. If you have a whole room in your house that you can dedicate to this, or a corner of a garage, sure. But don't buy a used Dell R610 1U server off eBay and expect to put it comfortably in your home office.
- vonseel 8y agoAgreed. Workstation towers like the Z420 are probably a lot quieter for small homelabs. I have a TS140 and Microserver, while the old Microserver is nearly silent, the TS140 can be irritating if seeking a dead silent room.
- vonseel 8y ago"After messing around with the ERL for months and going through flash drive after flash drive (What they boot from) I decided it was time to move on." Hmm, if he is talking about the EdgeRouter Lite, I have one, going strong since ~2014, didn't have any clue it has problems with flash drives, and there certainly isn't any need to open the case and mess with internal components. It's a good little router and I have few complaints with my Ubiquiti gear. I have not placed mine in a rack without adequate ventilation, however.
- DKnoll 8y agoI've heard the same complaint about the USB flash drives failing. I run an ER-8 but have had no issues. I think it may have been a bad batch or something. Some people have also had issues after installing OpenBSD on them without limiting writes.
- SteveNuts 8y agoI used to run a full size rack with servers and gear but ended up downsizing to three NUCs with 32GB in each, a simple managed Ubiquiti switch, and an 8 bay Synology like the one in the article. Doesn't have the "wow factor" of the big rack of equipment, but it's a lot easier to work on, quieter, and cheaper on the electric bill.
- dawnerd 8y agoI used to run a super loud HP server I got off craigslist for something like 70 bucks. Thing was a beast. Since have switched to a dual Xeon storinator with 30 8tb hdds. My next project is getting another storinator and finishing wiring the house for 10ge
- woolvalley 8y agoThe casual handgun on the desk in pictures is quite hilarious.
- choward 8y agoYou have to protect your data from both remote and local attacks.