4 ms·

Author here! Happy to take any questions or feedback about this post. I'm not an expert on computer hardware or building servers, but I feel like there aren't

Author here! Happy to take any questions or feedback about this post. I'm not an expert on computer hardware or building servers, but I feel like there aren't good resources for people at the intermediate level of choosing components for their home server, so I wrote this in the hopes that it might be useful to others interested in building a home VM server.

invisiblewasabi 6y ago
Thanks for taking questions! How did you go about picking the software? You could have picked xcp-ng, hyper-v, or just run a small Linux machine with QEMU-KVM that you could SSH in or VNC into and change. This would allow SCP or FTP capabilities for your ISOs and images. I've personally never used Proxmox, but I'd like to know how you ended up picking this one over other solutions. Why ditch ESXi? How do you end up managing using the same VMs? Do you have a template and you clone the template? Or do you have a way of just running both a template VM and an application VM, and have the app VM use the template VM for most of it's OS level needs. If you end up having 10 VMs, how do you properly update?
- mtlynch 6y ago
  Thanks for reading! >How did you go about picking the software? I wanted to go with something that was pretty mainstream. I don't care that much about cutting edge features, and I'm more interested in a mature product that I don't have to tinker much with. My experience with Kimchi made me more cautious about taking a chance on smaller projects. I keep wanting to like ESXi and not really enjoying it. I had a hard time scripting things with ESXi, and certain things seemed to require performing actions in the GUI. I can't remember at this point where the gap was, but I wasn't ever able to figure out how to clone a template and boot it up via CLI, whereas I found it easy to do that with Proxmox. >How do you end up managing using the same VMs? Do you have a template and you clone the template? Or do you have a way of just running both a template VM and an application VM, and have the app VM use the template VM for most of it's OS level needs. Yeah, I just have one pre-configured Ubuntu 20.04 template VM, and I clone that whenever I need a new VM. Occasionally, I'll boot up the template just to update its packages and tweak anything. I do all my VM management with Ansible, so there's a standard set of scripts that update and configure the machine whenever I clone a new one. >If you end up having 10 VMs, how do you properly update? I sometimes use Ansible to update everything, but honestly I get behind on updates. I wouldn't do that with an Internet-facing server, but none of these VMs are accessible outside my local network, so there's not really much attack surface.
justinclift 6y ago
An avenue that can be useful to explore - and my personal preference for home (quiet) gear - is professional workstations. They're often server grade gear, but in a tower form factor and made for low noise output. They can be pretty cheap on Ebay. For example, the Dell Precision 7810. Ebay (US) examples: https://www.ebay.com/itm/Dell-7810-Dual-Xeon-E5-2680-V4-32GB-Ram-512GB-SSD-3TB-HDD-2x-Quadro-600-Win-10/114474253218 https://www.ebay.com/itm/Dell-7810-Dual-Xeon-E5-2680-V4-32GB... https://www.ebay.com/itm/Precision-T7810-Dual-CPU-Xeon-Workstation-2X-E5-2620-V4-16-core-32threads/392997137030 https://www.ebay.com/itm/Precision-T7810-Dual-CPU-Xeon-Works... https://www.ebay.com/itm/Dell-Precision-T7810-Workstation-2-x-E5-2680-V4-2-4Ghz-16GB-RAM-1TB/333706728020 https://www.ebay.com/itm/Dell-Precision-T7810-Workstation-2-... Those look pretty good CPU wise, also having dual E5-2680's. They're the newer v4 revision though (more cores) and also leave plenty of room in the budget for upgrades (ram, ssd). --- If you specifically wanted server gear for some reason though, enterprise (server) gear generally have tower models. Careful research is needed though, to make sure you're getting something that isn't too noisy. ;) More dual Xeon Ebay (US) examples: https://www.ebay.com/itm/Dell-PowerEdge-T620-2x-Intel-Xeon-E5-2640-RAM-32GB-HD-6x300GB-OS-2008R2/154185895739 https://www.ebay.com/itm/Dell-PowerEdge-T620-2x-Intel-Xeon-E... https://www.ebay.com/itm/Dell-PowerEdge-T620-Server-Dual-6C-Xeon-E5-2640-2-5GHz-48GB-RAM-2x-PSUs/333776772450 https://www.ebay.com/itm/Dell-PowerEdge-T620-Server-Dual-6C-... Those both would leave plenty of room for upgrading the ram to 64GB, and (depending on their purchased options) may also come with remote KVM. Note that I haven't personally mucked around with Dell server gear in years, so no idea of their KVM situation these days. ;) For that model (PowerEdge T620) specifically, it's noise seems potentially workable after it gets past the first part of server boot: https://www.youtube.com/watch?v=UaQr2WFKq74 https://www.youtube.com/watch?v=UaQr2WFKq74 Note that although I'm just showing Dell gear here, HP, Lenovo, etc all have their own lines. eg HP Z840. With a bit of Ebay searching, it's possible to find some really good deals. :)
- mtlynch 6y ago
  Oh cool. I hadn't thought much about enterprise workstations, but those look pretty solid. I'll check those out on my next build. One downside for me is that I enjoy the process of picking out all the parts, and it seems like for a lot of these, you're buying a pre-packaged deal.
nix23 6y ago
Nice article and most important, have fun! BTW: Don't go to Reddit with projects like that, they are talkers no doers.
- mtlynch 6y ago
  Thanks for reading! Haha, yeah, there's a lot of noise on reddit, but they do often have legitimately good information.
lousken 6y ago
if motherboards were sparse why not look on ebay or something? this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI meaing you not only paid extra for wrong MB but also for the gpu and kvm i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country)
- mtlynch 6y ago
  Thanks for reading! >if motherboards were sparse why not look on ebay or something? I've never tried that. I thought that newegg had pretty good coverage for motherboards, and they make it easy to filter by CPU type, whereas I don't think eBay does. I just tried searching eBay for "intel 2011-v3 dual" and I did get >this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI I don't need audio output, so that doesn't make a difference to me. VGA output would be convenient, but it's also pretty easy to fix with a cheap GPU. IPMI would have been nice, and some readers have pointed out to me that SuperMicro has IPMI on a very similar board, so I should have gotten that one. >i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country) Does that assume the server runs under load 24/7? My CPUs are at very low load most of the time.
  - lousken 6y ago
    > >if motherboards were sparse why not look on ebay or something? > I've never tried that. I thought that newegg had pretty good coverage for motherboards, and they make it easy to filter by CPU type, whereas I don't think eBay does. I just tried searching eBay for "intel 2011-v3 dual" and I did get it's worth checking out supermicro site itself https://www.supermicro.com/en/products/motherboards/server-boards https://www.supermicro.com/en/products/motherboards/server-b... just select the platform processor series, check which board seems the best fit and then try if searching that on ebay yields any results > >this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI > I don't need audio output, so that doesn't make a difference to me. VGA output would be convenient, but it's also pretty easy to fix with a cheap GPU. IPMI would have been nice, and some readers have pointed out to me that SuperMicro has IPMI on a very similar board, so I should have gotten that one. I agree - you don't need audio output, but you've paid for it. That was my point. You could've saved on all three things by choosing the right motherboard. (edit: 3 things - integrated audio, dedicated gpu, kvm) > >i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country) > Does that assume the server runs under load 24/7? My CPUs are at very low load most of the time. basic PSU calculators can help you - https://outervision.com/power-supply-calculator https://outervision.com/power-supply-calculator -- just fiddle with cpu utilization and computer utilization time and you'll see also you might wanna check bios what mode you're running - if performance or eco or whatever (honestly not sure if workstation MBs are different in this regard, i am dealing with server MBs only) but you might be running on higher TDP than you think anyway

invisiblewasabi 6y ago

Thanks for taking questions! How did you go about picking the software? You could have picked xcp-ng, hyper-v, or just run a small Linux machine with QEMU-KVM that you could SSH in or VNC into and change. This would allow SCP or FTP capabilities for your ISOs and images. I've personally never used Proxmox, but I'd like to know how you ended up picking this one over other solutions. Why ditch ESXi? How do you end up managing using the same VMs? Do you have a template and you clone the template? Or do you have a way of just running both a template VM and an application VM, and have the app VM use the template VM for most of it's OS level needs. If you end up having 10 VMs, how do you properly update?

mtlynch 6y ago
Thanks for reading! >How did you go about picking the software? I wanted to go with something that was pretty mainstream. I don't care that much about cutting edge features, and I'm more interested in a mature product that I don't have to tinker much with. My experience with Kimchi made me more cautious about taking a chance on smaller projects. I keep wanting to like ESXi and not really enjoying it. I had a hard time scripting things with ESXi, and certain things seemed to require performing actions in the GUI. I can't remember at this point where the gap was, but I wasn't ever able to figure out how to clone a template and boot it up via CLI, whereas I found it easy to do that with Proxmox. >How do you end up managing using the same VMs? Do you have a template and you clone the template? Or do you have a way of just running both a template VM and an application VM, and have the app VM use the template VM for most of it's OS level needs. Yeah, I just have one pre-configured Ubuntu 20.04 template VM, and I clone that whenever I need a new VM. Occasionally, I'll boot up the template just to update its packages and tweak anything. I do all my VM management with Ansible, so there's a standard set of scripts that update and configure the machine whenever I clone a new one. >If you end up having 10 VMs, how do you properly update? I sometimes use Ansible to update everything, but honestly I get behind on updates. I wouldn't do that with an Internet-facing server, but none of these VMs are accessible outside my local network, so there's not really much attack surface.

justinclift 6y ago

An avenue that can be useful to explore - and my personal preference for home (quiet) gear - is professional workstations. They're often server grade gear, but in a tower form factor and made for low noise output. They can be pretty cheap on Ebay. For example, the Dell Precision 7810. Ebay (US) examples: https://www.ebay.com/itm/Dell-7810-Dual-Xeon-E5-2680-V4-32GB-Ram-512GB-SSD-3TB-HDD-2x-Quadro-600-Win-10/114474253218 https://www.ebay.com/itm/Dell-7810-Dual-Xeon-E5-2680-V4-32GB... https://www.ebay.com/itm/Precision-T7810-Dual-CPU-Xeon-Workstation-2X-E5-2620-V4-16-core-32threads/392997137030 https://www.ebay.com/itm/Precision-T7810-Dual-CPU-Xeon-Works... https://www.ebay.com/itm/Dell-Precision-T7810-Workstation-2-x-E5-2680-V4-2-4Ghz-16GB-RAM-1TB/333706728020 https://www.ebay.com/itm/Dell-Precision-T7810-Workstation-2-... Those look pretty good CPU wise, also having dual E5-2680's. They're the newer v4 revision though (more cores) and also leave plenty of room in the budget for upgrades (ram, ssd). --- If you specifically wanted server gear for some reason though, enterprise (server) gear generally have tower models. Careful research is needed though, to make sure you're getting something that isn't too noisy. ;) More dual Xeon Ebay (US) examples: https://www.ebay.com/itm/Dell-PowerEdge-T620-2x-Intel-Xeon-E5-2640-RAM-32GB-HD-6x300GB-OS-2008R2/154185895739 https://www.ebay.com/itm/Dell-PowerEdge-T620-2x-Intel-Xeon-E... https://www.ebay.com/itm/Dell-PowerEdge-T620-Server-Dual-6C-Xeon-E5-2640-2-5GHz-48GB-RAM-2x-PSUs/333776772450 https://www.ebay.com/itm/Dell-PowerEdge-T620-Server-Dual-6C-... Those both would leave plenty of room for upgrading the ram to 64GB, and (depending on their purchased options) may also come with remote KVM. Note that I haven't personally mucked around with Dell server gear in years, so no idea of their KVM situation these days. ;) For that model (PowerEdge T620) specifically, it's noise seems potentially workable after it gets past the first part of server boot: https://www.youtube.com/watch?v=UaQr2WFKq74 https://www.youtube.com/watch?v=UaQr2WFKq74 Note that although I'm just showing Dell gear here, HP, Lenovo, etc all have their own lines. eg HP Z840. With a bit of Ebay searching, it's possible to find some really good deals. :)

mtlynch 6y ago
Oh cool. I hadn't thought much about enterprise workstations, but those look pretty solid. I'll check those out on my next build. One downside for me is that I enjoy the process of picking out all the parts, and it seems like for a lot of these, you're buying a pre-packaged deal.

nix23 6y ago

Nice article and most important, have fun! BTW: Don't go to Reddit with projects like that, they are talkers no doers.

mtlynch 6y ago
Thanks for reading! Haha, yeah, there's a lot of noise on reddit, but they do often have legitimately good information.

lousken 6y ago

if motherboards were sparse why not look on ebay or something? this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI meaing you not only paid extra for wrong MB but also for the gpu and kvm i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country)

mtlynch 6y ago
Thanks for reading! >if motherboards were sparse why not look on ebay or something? I've never tried that. I thought that newegg had pretty good coverage for motherboards, and they make it easy to filter by CPU type, whereas I don't think eBay does. I just tried searching eBay for "intel 2011-v3 dual" and I did get >this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI I don't need audio output, so that doesn't make a difference to me. VGA output would be convenient, but it's also pretty easy to fix with a cheap GPU. IPMI would have been nice, and some readers have pointed out to me that SuperMicro has IPMI on a very similar board, so I should have gotten that one. >i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country) Does that assume the server runs under load 24/7? My CPUs are at very low load most of the time.
- lousken 6y ago
  > >if motherboards were sparse why not look on ebay or something? > I've never tried that. I thought that newegg had pretty good coverage for motherboards, and they make it easy to filter by CPU type, whereas I don't think eBay does. I just tried searching eBay for "intel 2011-v3 dual" and I did get it's worth checking out supermicro site itself https://www.supermicro.com/en/products/motherboards/server-boards https://www.supermicro.com/en/products/motherboards/server-b... just select the platform processor series, check which board seems the best fit and then try if searching that on ebay yields any results > >this motherboard seems like a really bad choice, because it's for a workstation not a server - doesn't have vga out, does have audio output and doesn't have IPMI > I don't need audio output, so that doesn't make a difference to me. VGA output would be convenient, but it's also pretty easy to fix with a cheap GPU. IPMI would have been nice, and some readers have pointed out to me that SuperMicro has IPMI on a very similar board, so I should have gotten that one. I agree - you don't need audio output, but you've paid for it. That was my point. You could've saved on all three things by choosing the right motherboard. (edit: 3 things - integrated audio, dedicated gpu, kvm) > >i'd also go for a better powersupply than bronze - at least gold, if that thing runs 24/7. The return of investment is like ~2.5yrs depending on the PSU (at least in my country) > Does that assume the server runs under load 24/7? My CPUs are at very low load most of the time. basic PSU calculators can help you - https://outervision.com/power-supply-calculator https://outervision.com/power-supply-calculator -- just fiddle with cpu utilization and computer utilization time and you'll see also you might wanna check bios what mode you're running - if performance or eco or whatever (honestly not sure if workstation MBs are different in this regard, i am dealing with server MBs only) but you might be running on higher TDP than you think anyway