12 ms·

Home Lab Hardware Guide

196 points by ashitlerferad 5y ago

zz865 5y ago
Would be great to have more VGA-IP solutions. My biggest problem is headless servers is when they dont boot its a PITA to move somewhere else to plug in a monitor and keyboard. Its the main reason I gave up and just hire a cloud host - even though its expensive.
- LargoLasskhyfv 5y ago
  https://www.aten.com/us/en/products/kvm/ https://www.aten.com/us/en/products/kvm/ ? Just as an example. There are much more, this was just from the top of my head, because they tend to just work.
  - zz865 5y ago
    KVM still needs a monitor which I'm trying to avoid.
aviatordongle 5y ago
When I had a larger homelab, I found it was a large proportion of my electricity usage. A constant 200W adds up, and 200W of heat is not desirable during Australian summers. I have settled on a PC Engine APU2 acting as my router, and a Raspberry Pi acting as my IoT brain. 10W. My other devices sleep.
atlgator 5y ago
Highly recommend the Discord on https://www.serverbuilds.net/ https://www.serverbuilds.net/. It's a great community built around home lab hardware using off-lease or decommissioned enterprise hardware. Guides range from 4U virtualization builds to custom pfSense routers.
deleted 5y ago
[deleted]
psim1 5y ago
Ikea Lack Rack for rack mounting on the cheap! https://boingboing.net/2020/08/14/lack-rack-ikeas-cheapest-ta.html https://boingboing.net/2020/08/14/lack-rack-ikeas-cheapest-t...
ncrmro 5y ago
6c/12t 64gb NUC with 1tb nvme drive works very nicely. I have offsite NAS but the nuc has a thunderbolt 3 port for a raid array or 10 GB ethernet if needed. With KVM on Ubuntu 21 zfs root pretty much covers most of these home lab uses. Have a Linux workstation that also does gpu pass through for windows gaming. Use a physical Kvm to switch between hosts. Really hard to justify a rack these days
deleted 5y ago
[deleted]
sgarland 5y ago
This is a good, if not opinionated, guide. And to be fair, r/homelab and its ilk are so full of options that it's easy to become overwhelmed. Personally, I settled on Supermicro because they're modular, and don't care what brand of stuff you throw in them (HP is notorious for spinning fans to turbo if you put non-HP disks into them), although I may be picking up two Dells to complement the one I have for a Proxmox HA/Ceph cluster.
- whalesalad 5y ago
  I love Dell. IDRAC is killer.
  - sgarland 5y ago
    The HTML5 interface is definitely very nice. I haven't had a chance to use Supermicro's HTML5 IPMI, as I have an X9 board, and to my knowledge the minimum support for it is X10.
    - sjackso 5y ago
      I have a little experience with both the old and new Supermicro stuff. The new x10 IPMI experience is a lot like the old x9 java app experience, except you don't have to dig out an ancient computer to make it work. (Which feels great by comparison!)
- croutonwagon 5y ago
  I dont run server anything. I have dell 7050's running a VMware/virtualization lab. They can have 64GB of ram and can handle anything i throw at them. Not iKVM though. But i just walk across the room and hook up a monitor the one or two times a year i need to. Honestly Synology has been the best godsend. I was a SAN admin in a previous life. I have run freenas, openfiler, openfiler in HA, linux+NFS+iSCSI etc over the years. Synology generally makes it simple and totally integrated and allows me to play with other things rather than getting storage working.
  - 5y ago
willis936 5y ago
I'm on board with most of this except the suggestion that rack mounted hardware should be kept where people live. No. I can't afford a house but like to tinker with networking. I pay more for weaker equipment so it will be low power/low noise. I'd love to just buy half a dozen used Dell PowerEdges but rack-mounted hardware is insanely loud. A basement is the ideal spot. Water isn't an issue as long as your rack is not directly under anything that could leak (including on higher floors) and has a palette underneath it. If there is the possibility of your basement flooding more than 2 inches then you have bigger problems you need to address first. Keeping a rack with electronic equipment there will motivate you to do what you should be doing to the place anyway: dehumidifying and managing pests.
- jjav 5y ago
  > but rack-mounted hardware is insanely loud A 1U chassis stuffed with high power dissipation components will be insanely loud like a jet engine. But it's easy to avoid that. The key for quiet is to use 4U chassis for anything that requires significant cooling so it has large fans. On my home rack I have three 1U computers, two are Atom-based (fanless with SSD, so zero noise) and one is a Celeron-based with slow fans (about as loud as my MacBook). The larger machine is a 4U with larger fans, so same sound level as any tower case (a 4U is literally a tower case sideways, with rack mounting tabs).
- croutonwagon 5y ago
  Yeah i tend to agree.. I had a small rack in our old house. Mostly just to house the router/switch because the little cabinet wouldnt fit. My wife dubbed it the EyeRack, short for eyesore. For the most part you can easily hide this stuff in a cabinet or a bookshelf, no one will be the wiser. Ill never really understand people using pizza box style servers, especially 1U units, in a home. With the sole exception of the one time i saw one stood veritcally behind an entertainment center. I think i was the only person at the party to notice it.
- bluedino 5y ago
  IF you live near a co-lo, you might be able to get a half rack and go in on it with a buddy or two.
KaiserPro 5y ago
For UK people https://www.bargainhardware.co.uk/ https://www.bargainhardware.co.uk/ is an _excellent_ source of kit. Personally I steer away from cisco. Yes, some people in enterprise swear by it, but I _personally_ hate it with a passion. However there is a fucktonne of it on ebay. I use ubiquiti for APs, but I've not tried their switching. Currently I have some dlink "smart" stuff. Its PoE and has vlans, which is good enough for my purposes. Can do 10gig, not bad for < £120 (second hand) Firewall, I'm all for pfsense. I've never liked hardware firewall/router appliances. They've always sucked.
rektide 5y ago
> AMD has really raised the bar. I’m most impressed with the CPU performance of the M715q. They both run quiet and cool, with Ubuntu Server and Windows 10. The M715q was offered with a fantastic 4750G chip, a Ryzen 7 Pro chip with 8 core. Today all one can buy in terms of small form factor business PCs is an M75n with low end low power Ryzen 3300U, a multiple-generations old Ryzen 3 with 4 cores. Small business PCs are great, and for a while, there was serious excitement that AMD was going to make this segment much more interesting. Those dreams seem to have all been cancelled. I'm glad to see that affordable, competent AMD laptops are about, because in many ways it feels like AMD has succeeded so greatly that they have vanished from the market. They don't seem to be allocating production capacity to consumer GPUs, they seem to have withdrawn from this price-conscious market segment,... AMD keeps vanishing.
the_third_wave 5y ago
I made a rack out of some dumpster-dived supermarket shelves, lumber, a truck air filter and a forced draft fan. The thing doubles as drying cabinet for produce (mint, mushrooms, fruit etc.) by having the equipment in the top half of the rack followed by an air flow divider and 8 rack-sized metal-mesh-covered drying frames. From top to bottom the thing contains: * D-Link DGS-3324SR (managed switch, €35) * HP DL380G7 with 2xX5675 @3.07GHz, 128GB (ECC) RAM and 8x147GB SAS drives (€450) * NetApp DS4243 (24x3.5” SAS array, currently populated with 24x650GB 15K SAS drives, €400) * the mentioned airflow divider * 8 drying frames It is managed through Proxmox on Debian and runs a host of services including a virtual router (OpenWRT), serving us here on the farm and the extended family spread over 2 countries. The server-mounted array is used as a boot drive and to host some container and VM images, the DS4243 array is configured as a JBOD running a mixture of LVM/mdadm managed arrays and stripe sets used as VM/container image and data storage. I chose mdadm over ZFS because of the greater flexibility it offers. The array in the DL380 is managed by the P410i array controller (i.e. hardware raid), I have 4 spare drives in storage to be used as replacements for failed drives. The rack is about 1.65m high, it looks like this (here minus the DS4243 array which now sits just above the air flow divider): https://imgur.com/a/M4Lbf1K https://imgur.com/a/M4Lbf1K In the not-too-distant future I’ll replace the 15K SAS drives with larger albeit slower (7.2K) SAS or SATA drives to get more space and (especially) less heat - those 15K drives run hot. After a warm summer I added an extra air intake + filter on the front side (not visible on the photos), facing the equipment. This is made possible by the fact that cooling air is pulled through the contraption from the underside instead of being blown in through the filter(s). I chose this specific hardware - a fairly loaded DL380G7, the DS4243 - because these offered the best price/performance ratio when I got them (in 2018). Spare parts for these devices are cheap and easily available, I made sure to get a full complement of power supplies for both devices (2 for the DL380G7, 4 for the DS4243) although I’m only using half of these. I recently had to replace a power supply in the DL380 (€20) and two drives in the DS4243 (€20/piece), for the rest everything has been working fine for close to 4 years now. On the question whether this much hardware is needed, well, that depends on what you want to do. If you just want to serve media files and have a shell host to log in to the answer is probably ‘no’, depending on the size of the library. Instead of using ‘enterprise class’ equipment you could try to build a system tailored to the home environment which prioritizes a reduction in power consumption and noise levels over redundancy and performance. You’ll probably end up spending about the same amount of money for hardware, a bit more in time and get a substantially lower performing system but you’d be rewarded by the lower noise levels and reduced power consumption. The latter can be offset by adding a few solar panels, the former by moving the rack to a less noise-sensitive location - the basement, the barn, etc. As to having 19" rack equipment in the home I'd say this is feasible as long as you don't have to sit right next to the things. Even with the totally enclosed, forced-draft rack I made the thing does produce enough noise to make it hard to forget it is there.
walterbell 5y ago
> Enterprise features: Ubiquiti EdgeRouter ER-10X, 10 Port Gigabit Router with PoE Flexibility – $110 (specs) – (10) Gigabit RJ45 Ports, PoE Passthrough on Port 10, Dual-Core, 880 MHz, MIPS1004Kc Processor, 512 MB DDR3 RAM, 512 MB NAND Flash Storage, Internal Switch, Serial Console Port Has anyone been able to purchase a small Ubiquiti EdgeRouter in the last six months? They've been out of stock at Amazon, Newegg, B&H. Beginning to wonder if they have deprioritized the consumer market, since other vendors are shipping routers.
- rafaelm 5y ago
  I was hunting the 8 port switch for a while. I had to setup my own stock monitoring with alerts and managed to catch one that way.
- psim1 5y ago
  Yes, I bought an ER-4 last month from NewEgg. In fact, it is in stock and has a promo code for $10 off currently. https://www.newegg.com/p/0XM-0013-00087 https://www.newegg.com/p/0XM-0013-00087
- tbyehl 5y ago
  Ubiquiti appears to have been prioritizing their own online store since the supply chain disruptions began. The ER-10X in particular may also be suffering from a general lack of popularity -- they've only had stock twice this year in fairly low quantities. They sell through fairly quick, but not nearly as fast as the ER-X which has been stocked regularly and in much larger quantities. There's an inventory tracker for the Ubiquiti store on the Discord. https://discord.gg/ui https://discord.gg/ui
- mbreese 5y ago
  Are they still making them? I thought they had switched to pushing their Dream Machine as opposed to the Edge Router series.
  - nullwarp 5y ago
    I _hate_ the Dream Machines. We've been switching to them at work and the whole cloud UI is just an absolute mess. It's so hard to find anything. I will be sad when the last of our Mikrotik stuff gets swapped out.
youngtaff 5y ago
Serve the Home did a fab set of reviews on the different SFF machines and how useful they are for a homelab https://www.servethehome.com/?s=tinyminimicro https://www.servethehome.com/?s=tinyminimicro
tbyehl 5y ago
I've been re-establishing my home lab and decided to get away from rackmount gear. I found ServeTheHome's TinyMiniMicro[1] series invaluable for choosing some mini PCs that would be right for me. I went with three HP Prodesk 600 G4 that averaged $250/ea with the i5-8500T/i5-8600T, 256GB NVMe, and a total of 40GB RAM. They can go to 64GB RAM, the dual M.2 M-key plus potentially an SFF SATA drive offer plenty of storage potential, they're effectively silent, and power consumption is much lower than a big server full of fans. vPro potentially offers out-of-band remote management but I haven't tried digging into that yet. I have two dedicated to Frigate with M.2 Coral TPUs. On the third I've been consolidating the sprawl of Linux VMs and Docker containers running home automation and network management stuff. Could probably make do with just two but why buy only two when you can have three? [1] https://www.servethehome.com/tag/tinyminimicro/ https://www.servethehome.com/tag/tinyminimicro/
KingMachiavelli 5y ago
Rack mount hardware is almost always expensive, loud, and power hungry. I just have never seen the point of building a home lab like this. A single ATX desktop can do almost [1] everything a homelab at a fraction of the cost & power consuption. I think a lot of the reason for homelabs/server hardware was to get access to more CPU cores, now that 8+ cores are very cheap it is actually cheaper to buy a new consumer desktop than it is to run an old server. What makes even less sense is wanting use to use software like vSphere or ESXi since its about 10x more complicated than just using virt-manager/QEMU. It's like using an excavator to dig a fire pit. Server hardware & software makes sense when it's not your home because then you do need a remote access tool like iDRACK. (There are DIY options if you just need something for personal use). That said if you enjoy it as a hobby (or your homelab is actually a business thing) then go for it.
- neilv 5y ago
  Absolutely, rackmount servers tend to be very loud and often power-hungry. But if you think of rackmount as a form factor, it can make some practical and aesthetic sense. The IKEA CORRAS Birch Effect "rack cabinet" in my living room currently has a nice virtually silent GPU compute server based on a consumer/gaming PC in a 4U short-depth rackmount case, a mostly passively-cooled Supermicro Atom server in which I've replaced the PSU fan with a Noctua, and a sinewave UPS. (There's an ongoing evolution of gear over time, including more exotic stuff, and most toys I might play with in the future will also fit this form factor.) Aesthetically, those stack on the bottom of the cabinet, a Birch Effect shelf sits on top, and the plastic OpenWrt WiFi router with all the pokey antennae sits safely in the shelf cubbyhole. (Once I need detached WiFi APs, I'll probably build a 1U or 2U router with opnSense/Linux/*BSD. And I have a discreet slide-out rackmount console for if I ever get a deeper cabinet.) I like to think it looks like understated black home AV that's not out of place in a living room, which is better than a tangle of assorted non-rackmount PCs and UPS. Practically, besides the tidy organizing and uniform cooling airflow, I have some rack posts I can use if the stack of gear gets too large/tricky for pulling individual boxes without disturbing the others too much. Also, when I lived in a dodgier student apartment, fastening a bunch of rackmount gear together with security-head screws seemed a good way to discourage a burglar from walking off with my data.
- 5y ago
unethical_ban 5y ago
I just did some work on my main server, and my take is a bit different. I upgraded my desktop and built a server from a Ryzen 1700, put 64GB of RAM in it, and now this one device acts as DNS filter/cache(Pi-Hole), VPN server (Pi-VPN Wireguard), and a 10TB ZFS NAS. This is just the base, I also use it for gaming and labs. The main recommendations: Fractal Design Define R5 - this is a large case, and is pretty wide - but it is a dream to work in. The extra width gives plenty of room behind the motherboard for hiding cables. It has quiet fans, it is built to minimize noise, and it can hold 8+ hard drives. OS: Proxmox. I use this as the host OS, and configure my ZFS on the host. I then expose the ZFS as a NAS via a privileged container running Turnkey Linux. If you get some multi-port NICs on it, you can put an OPNSense firewall as a VM, and use the machine as your router as well. In the end, you would only need UPS, modem, small switch, and the host.
dragontamer 5y ago
Note that 10Gbe SFP+ switches have come down to $250 or so, and may be worthwhile for homelabs to experiment with. See Mikrotik's CRS309-1G-8S+IN, or servethehome's review (https://www.servethehome.com/mikrotik-crs309-1g-8sin-review-inexpensive-8x-10gbe-switch/ https://www.servethehome.com/mikrotik-crs309-1g-8sin-review-...). If only because most of us probably already know how to use Cat5/Cat6 Ethernet, but how many of us have experimented with fiber optics? 10Gb Ethernet over Cat6 exists too. But that may be boring for some! Home labs are about experimenting with new things.
- LargoLasskhyfv 5y ago
  https://fs.com https://fs.com for all your optics! Maybe not all, but very usable and informative regarding prices and availabilty of all that 'exotic' stuff.
- floatboth 5y ago
  Aruba 1930 is a great option too
- bombcar 5y ago
  Another advantage of fiber is it helps prevent lightning and other power surges from spreading. If your equipment is protected on the power edge, fiber isolates it on the network side.
  - dragontamer 5y ago
    Hmm, maybe "fiber" is the wrong moniker here. I'm more talking about SFP+ ports, because most of your connections within the rack will probably be DAC (copper cables pretending to be fiber) for lower costs. Fiber is really for longer runs. If you only have a few feet worth of cable, I'm not sure if fiber per se is worth it over DAC. But learning to work with SFP+ hardware is a skill, just like learning to strip CAT6 cable or run it around. Working with DAC cables, or SFP+ modules and finding what works is the "dumb part" of IT, but the kind of stuff you need to practice a few times to understand. ---- Grabbing a few SFP+ ConnectX-2 cards from Ebay (for $30 or so), a few DAC cables, and a $250 switch... you can be well on your way to a 10Gbit network.
memetomancer 5y ago
So many of these posts gainsaying the practice of a home lab are spot on. In my view it's plain foolish to try and cram enterprise gear into a living space. It's almost too hot and loud for my office, why would I take any of that home? The other posts talking about Tiny Mini Micros are on the right track but I think it goes further yet - there's good reason to have a small rack of crap in the corner: - ISP hardware. - pfSense gateway. - Wifi base station. - A good gigabit switch for the house. - Those tiny mini micros or Mac minis for lab stuff. - a NAS chassis or two. - Raspberry Pi clusters. - PiDP-11 or other such hobby stuff that needs a place to sit and blink. There are plenty of other uses too, like security DVRs, ingest stations for cameras/recorders, optical and tape media devices, etc. None of that stuff is hot or loud, but you probably wouldn't want it piled up on your desk or spilling out of some bookshelf. And I think the article kinda gets at that point, tbh.
ChuckMcM 5y ago
I strongly endorse this notion of equipping your own laboratory for your experiments. Learning through doing is always more durable than learning through reading only. While the author is looking at learning about and perfecting their skills as an administrator of networked computer systems there are other "kinds" of laboratories that people set up. Mine, and one I'm more familiar with, are electronics labs. If you're going to be learning about circuits and such it helps to have the basic kit at hand. Similarly for people doing robotics, having a 3D printer in their home lab is essential these days. Nearly everything you might do in a home laboratory will involve some sort of data processing so the ideas by the author are great for creating the lab's "IT infrastructure." In California it also makes it easier to defend the "I built this technology my own gear (picture/description of lab) so you don't own it." But that may be unique to California.
- flapjaxy 5y ago
  Exactly, I think this thread is focusing on the utility of an environment from a mature SDec POV. Personally having a cheap R710 allowed me to practice my sysadmin, networking, security and a host of other skills that directly translated to my (then) current and future roles. I think the perception of a virtualized environment being the same is false and diminishes the sys/network/security admin's knowledge set. It's is akin to saying that writing JS is the same as Java, coding is coding. I'll also say the number of times I've been in rooms of IT pros who don't understand underlying systems, OS, or networking is disturbingly high. Lastly, even if it is frivolous I see tinkering with hardware as part of the hacking spirit. It's fun, I like it and it gives me something to grumble about with the grey beards.
  - ChuckMcM 5y ago
    Here in the Bay Area I know a guy who both wires and de-wires commercial buildings for networking. It is surprising to me how many businesses move out and simply discard any equipment they installed in the POE to support their network. He gave me a Catalyst 9000 series to play with (he usually sells them to the resellers) and it was cool to be playing with "real" hardware but man, that thing was LOUD with the fans.
- 5y ago
User23 5y ago
UPS plus a generator is a must have if you live someplace with severe weather.

196 points by ashitlerferad 5y ago

Would be great to have more VGA-IP solutions. My biggest problem is headless servers is when they dont boot its a PITA to move somewhere else to plug in a monitor and keyboard. Its the main reason I gave up and just hire a cloud host - even though its expensive.

LargoLasskhyfv 5y ago
https://www.aten.com/us/en/products/kvm/ https://www.aten.com/us/en/products/kvm/ ? Just as an example. There are much more, this was just from the top of my head, because they tend to just work.
- zz865 5y ago
  KVM still needs a monitor which I'm trying to avoid.

aviatordongle 5y ago

When I had a larger homelab, I found it was a large proportion of my electricity usage. A constant 200W adds up, and 200W of heat is not desirable during Australian summers. I have settled on a PC Engine APU2 acting as my router, and a Raspberry Pi acting as my IoT brain. 10W. My other devices sleep.

atlgator 5y ago

Highly recommend the Discord on https://www.serverbuilds.net/ https://www.serverbuilds.net/. It's a great community built around home lab hardware using off-lease or decommissioned enterprise hardware. Guides range from 4U virtualization builds to custom pfSense routers.

deleted 5y ago

[deleted]

psim1 5y ago

Ikea Lack Rack for rack mounting on the cheap! https://boingboing.net/2020/08/14/lack-rack-ikeas-cheapest-ta.html https://boingboing.net/2020/08/14/lack-rack-ikeas-cheapest-t...

ncrmro 5y ago

6c/12t 64gb NUC with 1tb nvme drive works very nicely. I have offsite NAS but the nuc has a thunderbolt 3 port for a raid array or 10 GB ethernet if needed. With KVM on Ubuntu 21 zfs root pretty much covers most of these home lab uses. Have a Linux workstation that also does gpu pass through for windows gaming. Use a physical Kvm to switch between hosts. Really hard to justify a rack these days

deleted 5y ago

[deleted]

sgarland 5y ago

This is a good, if not opinionated, guide. And to be fair, r/homelab and its ilk are so full of options that it's easy to become overwhelmed. Personally, I settled on Supermicro because they're modular, and don't care what brand of stuff you throw in them (HP is notorious for spinning fans to turbo if you put non-HP disks into them), although I may be picking up two Dells to complement the one I have for a Proxmox HA/Ceph cluster.

whalesalad 5y ago
I love Dell. IDRAC is killer.
- sgarland 5y ago
  The HTML5 interface is definitely very nice. I haven't had a chance to use Supermicro's HTML5 IPMI, as I have an X9 board, and to my knowledge the minimum support for it is X10.
  - sjackso 5y ago
    I have a little experience with both the old and new Supermicro stuff. The new x10 IPMI experience is a lot like the old x9 java app experience, except you don't have to dig out an ancient computer to make it work. (Which feels great by comparison!)
croutonwagon 5y ago
I dont run server anything. I have dell 7050's running a VMware/virtualization lab. They can have 64GB of ram and can handle anything i throw at them. Not iKVM though. But i just walk across the room and hook up a monitor the one or two times a year i need to. Honestly Synology has been the best godsend. I was a SAN admin in a previous life. I have run freenas, openfiler, openfiler in HA, linux+NFS+iSCSI etc over the years. Synology generally makes it simple and totally integrated and allows me to play with other things rather than getting storage working.
- 5y ago

willis936 5y ago

I'm on board with most of this except the suggestion that rack mounted hardware should be kept where people live. No. I can't afford a house but like to tinker with networking. I pay more for weaker equipment so it will be low power/low noise. I'd love to just buy half a dozen used Dell PowerEdges but rack-mounted hardware is insanely loud. A basement is the ideal spot. Water isn't an issue as long as your rack is not directly under anything that could leak (including on higher floors) and has a palette underneath it. If there is the possibility of your basement flooding more than 2 inches then you have bigger problems you need to address first. Keeping a rack with electronic equipment there will motivate you to do what you should be doing to the place anyway: dehumidifying and managing pests.

jjav 5y ago
> but rack-mounted hardware is insanely loud A 1U chassis stuffed with high power dissipation components will be insanely loud like a jet engine. But it's easy to avoid that. The key for quiet is to use 4U chassis for anything that requires significant cooling so it has large fans. On my home rack I have three 1U computers, two are Atom-based (fanless with SSD, so zero noise) and one is a Celeron-based with slow fans (about as loud as my MacBook). The larger machine is a 4U with larger fans, so same sound level as any tower case (a 4U is literally a tower case sideways, with rack mounting tabs).
croutonwagon 5y ago
Yeah i tend to agree.. I had a small rack in our old house. Mostly just to house the router/switch because the little cabinet wouldnt fit. My wife dubbed it the EyeRack, short for eyesore. For the most part you can easily hide this stuff in a cabinet or a bookshelf, no one will be the wiser. Ill never really understand people using pizza box style servers, especially 1U units, in a home. With the sole exception of the one time i saw one stood veritcally behind an entertainment center. I think i was the only person at the party to notice it.
bluedino 5y ago
IF you live near a co-lo, you might be able to get a half rack and go in on it with a buddy or two.

KaiserPro 5y ago

For UK people https://www.bargainhardware.co.uk/ https://www.bargainhardware.co.uk/ is an _excellent_ source of kit. Personally I steer away from cisco. Yes, some people in enterprise swear by it, but I _personally_ hate it with a passion. However there is a fucktonne of it on ebay. I use ubiquiti for APs, but I've not tried their switching. Currently I have some dlink "smart" stuff. Its PoE and has vlans, which is good enough for my purposes. Can do 10gig, not bad for < £120 (second hand) Firewall, I'm all for pfsense. I've never liked hardware firewall/router appliances. They've always sucked.

rektide 5y ago

> AMD has really raised the bar. I’m most impressed with the CPU performance of the M715q. They both run quiet and cool, with Ubuntu Server and Windows 10. The M715q was offered with a fantastic 4750G chip, a Ryzen 7 Pro chip with 8 core. Today all one can buy in terms of small form factor business PCs is an M75n with low end low power Ryzen 3300U, a multiple-generations old Ryzen 3 with 4 cores. Small business PCs are great, and for a while, there was serious excitement that AMD was going to make this segment much more interesting. Those dreams seem to have all been cancelled. I'm glad to see that affordable, competent AMD laptops are about, because in many ways it feels like AMD has succeeded so greatly that they have vanished from the market. They don't seem to be allocating production capacity to consumer GPUs, they seem to have withdrawn from this price-conscious market segment,... AMD keeps vanishing.

the_third_wave 5y ago

I made a rack out of some dumpster-dived supermarket shelves, lumber, a truck air filter and a forced draft fan. The thing doubles as drying cabinet for produce (mint, mushrooms, fruit etc.) by having the equipment in the top half of the rack followed by an air flow divider and 8 rack-sized metal-mesh-covered drying frames. From top to bottom the thing contains: * D-Link DGS-3324SR (managed switch, €35) * HP DL380G7 with 2xX5675 @3.07GHz, 128GB (ECC) RAM and 8x147GB SAS drives (€450) * NetApp DS4243 (24x3.5” SAS array, currently populated with 24x650GB 15K SAS drives, €400) * the mentioned airflow divider * 8 drying frames It is managed through Proxmox on Debian and runs a host of services including a virtual router (OpenWRT), serving us here on the farm and the extended family spread over 2 countries. The server-mounted array is used as a boot drive and to host some container and VM images, the DS4243 array is configured as a JBOD running a mixture of LVM/mdadm managed arrays and stripe sets used as VM/container image and data storage. I chose mdadm over ZFS because of the greater flexibility it offers. The array in the DL380 is managed by the P410i array controller (i.e. hardware raid), I have 4 spare drives in storage to be used as replacements for failed drives. The rack is about 1.65m high, it looks like this (here minus the DS4243 array which now sits just above the air flow divider): https://imgur.com/a/M4Lbf1K https://imgur.com/a/M4Lbf1K In the not-too-distant future I’ll replace the 15K SAS drives with larger albeit slower (7.2K) SAS or SATA drives to get more space and (especially) less heat - those 15K drives run hot. After a warm summer I added an extra air intake + filter on the front side (not visible on the photos), facing the equipment. This is made possible by the fact that cooling air is pulled through the contraption from the underside instead of being blown in through the filter(s). I chose this specific hardware - a fairly loaded DL380G7, the DS4243 - because these offered the best price/performance ratio when I got them (in 2018). Spare parts for these devices are cheap and easily available, I made sure to get a full complement of power supplies for both devices (2 for the DL380G7, 4 for the DS4243) although I’m only using half of these. I recently had to replace a power supply in the DL380 (€20) and two drives in the DS4243 (€20/piece), for the rest everything has been working fine for close to 4 years now. On the question whether this much hardware is needed, well, that depends on what you want to do. If you just want to serve media files and have a shell host to log in to the answer is probably ‘no’, depending on the size of the library. Instead of using ‘enterprise class’ equipment you could try to build a system tailored to the home environment which prioritizes a reduction in power consumption and noise levels over redundancy and performance. You’ll probably end up spending about the same amount of money for hardware, a bit more in time and get a substantially lower performing system but you’d be rewarded by the lower noise levels and reduced power consumption. The latter can be offset by adding a few solar panels, the former by moving the rack to a less noise-sensitive location - the basement, the barn, etc. As to having 19" rack equipment in the home I'd say this is feasible as long as you don't have to sit right next to the things. Even with the totally enclosed, forced-draft rack I made the thing does produce enough noise to make it hard to forget it is there.

walterbell 5y ago

> Enterprise features: Ubiquiti EdgeRouter ER-10X, 10 Port Gigabit Router with PoE Flexibility – $110 (specs) – (10) Gigabit RJ45 Ports, PoE Passthrough on Port 10, Dual-Core, 880 MHz, MIPS1004Kc Processor, 512 MB DDR3 RAM, 512 MB NAND Flash Storage, Internal Switch, Serial Console Port Has anyone been able to purchase a small Ubiquiti EdgeRouter in the last six months? They've been out of stock at Amazon, Newegg, B&H. Beginning to wonder if they have deprioritized the consumer market, since other vendors are shipping routers.

rafaelm 5y ago
I was hunting the 8 port switch for a while. I had to setup my own stock monitoring with alerts and managed to catch one that way.
psim1 5y ago
Yes, I bought an ER-4 last month from NewEgg. In fact, it is in stock and has a promo code for $10 off currently. https://www.newegg.com/p/0XM-0013-00087 https://www.newegg.com/p/0XM-0013-00087
tbyehl 5y ago
Ubiquiti appears to have been prioritizing their own online store since the supply chain disruptions began. The ER-10X in particular may also be suffering from a general lack of popularity -- they've only had stock twice this year in fairly low quantities. They sell through fairly quick, but not nearly as fast as the ER-X which has been stocked regularly and in much larger quantities. There's an inventory tracker for the Ubiquiti store on the Discord. https://discord.gg/ui https://discord.gg/ui
mbreese 5y ago
Are they still making them? I thought they had switched to pushing their Dream Machine as opposed to the Edge Router series.
- nullwarp 5y ago
  I _hate_ the Dream Machines. We've been switching to them at work and the whole cloud UI is just an absolute mess. It's so hard to find anything. I will be sad when the last of our Mikrotik stuff gets swapped out.

youngtaff 5y ago

Serve the Home did a fab set of reviews on the different SFF machines and how useful they are for a homelab https://www.servethehome.com/?s=tinyminimicro https://www.servethehome.com/?s=tinyminimicro

tbyehl 5y ago

I've been re-establishing my home lab and decided to get away from rackmount gear. I found ServeTheHome's TinyMiniMicro[1] series invaluable for choosing some mini PCs that would be right for me. I went with three HP Prodesk 600 G4 that averaged $250/ea with the i5-8500T/i5-8600T, 256GB NVMe, and a total of 40GB RAM. They can go to 64GB RAM, the dual M.2 M-key plus potentially an SFF SATA drive offer plenty of storage potential, they're effectively silent, and power consumption is much lower than a big server full of fans. vPro potentially offers out-of-band remote management but I haven't tried digging into that yet. I have two dedicated to Frigate with M.2 Coral TPUs. On the third I've been consolidating the sprawl of Linux VMs and Docker containers running home automation and network management stuff. Could probably make do with just two but why buy only two when you can have three? [1] https://www.servethehome.com/tag/tinyminimicro/ https://www.servethehome.com/tag/tinyminimicro/

KingMachiavelli 5y ago

Rack mount hardware is almost always expensive, loud, and power hungry. I just have never seen the point of building a home lab like this. A single ATX desktop can do almost [1] everything a homelab at a fraction of the cost & power consuption. I think a lot of the reason for homelabs/server hardware was to get access to more CPU cores, now that 8+ cores are very cheap it is actually cheaper to buy a new consumer desktop than it is to run an old server. What makes even less sense is wanting use to use software like vSphere or ESXi since its about 10x more complicated than just using virt-manager/QEMU. It's like using an excavator to dig a fire pit. Server hardware & software makes sense when it's not your home because then you do need a remote access tool like iDRACK. (There are DIY options if you just need something for personal use). That said if you enjoy it as a hobby (or your homelab is actually a business thing) then go for it.

neilv 5y ago
Absolutely, rackmount servers tend to be very loud and often power-hungry. But if you think of rackmount as a form factor, it can make some practical and aesthetic sense. The IKEA CORRAS Birch Effect "rack cabinet" in my living room currently has a nice virtually silent GPU compute server based on a consumer/gaming PC in a 4U short-depth rackmount case, a mostly passively-cooled Supermicro Atom server in which I've replaced the PSU fan with a Noctua, and a sinewave UPS. (There's an ongoing evolution of gear over time, including more exotic stuff, and most toys I might play with in the future will also fit this form factor.) Aesthetically, those stack on the bottom of the cabinet, a Birch Effect shelf sits on top, and the plastic OpenWrt WiFi router with all the pokey antennae sits safely in the shelf cubbyhole. (Once I need detached WiFi APs, I'll probably build a 1U or 2U router with opnSense/Linux/*BSD. And I have a discreet slide-out rackmount console for if I ever get a deeper cabinet.) I like to think it looks like understated black home AV that's not out of place in a living room, which is better than a tangle of assorted non-rackmount PCs and UPS. Practically, besides the tidy organizing and uniform cooling airflow, I have some rack posts I can use if the stack of gear gets too large/tricky for pulling individual boxes without disturbing the others too much. Also, when I lived in a dodgier student apartment, fastening a bunch of rackmount gear together with security-head screws seemed a good way to discourage a burglar from walking off with my data.
5y ago

unethical_ban 5y ago

I just did some work on my main server, and my take is a bit different. I upgraded my desktop and built a server from a Ryzen 1700, put 64GB of RAM in it, and now this one device acts as DNS filter/cache(Pi-Hole), VPN server (Pi-VPN Wireguard), and a 10TB ZFS NAS. This is just the base, I also use it for gaming and labs. The main recommendations: Fractal Design Define R5 - this is a large case, and is pretty wide - but it is a dream to work in. The extra width gives plenty of room behind the motherboard for hiding cables. It has quiet fans, it is built to minimize noise, and it can hold 8+ hard drives. OS: Proxmox. I use this as the host OS, and configure my ZFS on the host. I then expose the ZFS as a NAS via a privileged container running Turnkey Linux. If you get some multi-port NICs on it, you can put an OPNSense firewall as a VM, and use the machine as your router as well. In the end, you would only need UPS, modem, small switch, and the host.

dragontamer 5y ago

Note that 10Gbe SFP+ switches have come down to $250 or so, and may be worthwhile for homelabs to experiment with. See Mikrotik's CRS309-1G-8S+IN, or servethehome's review (https://www.servethehome.com/mikrotik-crs309-1g-8sin-review-inexpensive-8x-10gbe-switch/ https://www.servethehome.com/mikrotik-crs309-1g-8sin-review-...). If only because most of us probably already know how to use Cat5/Cat6 Ethernet, but how many of us have experimented with fiber optics? 10Gb Ethernet over Cat6 exists too. But that may be boring for some! Home labs are about experimenting with new things.

LargoLasskhyfv 5y ago
https://fs.com https://fs.com for all your optics! Maybe not all, but very usable and informative regarding prices and availabilty of all that 'exotic' stuff.
floatboth 5y ago
Aruba 1930 is a great option too
bombcar 5y ago
Another advantage of fiber is it helps prevent lightning and other power surges from spreading. If your equipment is protected on the power edge, fiber isolates it on the network side.
- dragontamer 5y ago
  Hmm, maybe "fiber" is the wrong moniker here. I'm more talking about SFP+ ports, because most of your connections within the rack will probably be DAC (copper cables pretending to be fiber) for lower costs. Fiber is really for longer runs. If you only have a few feet worth of cable, I'm not sure if fiber per se is worth it over DAC. But learning to work with SFP+ hardware is a skill, just like learning to strip CAT6 cable or run it around. Working with DAC cables, or SFP+ modules and finding what works is the "dumb part" of IT, but the kind of stuff you need to practice a few times to understand. ---- Grabbing a few SFP+ ConnectX-2 cards from Ebay (for $30 or so), a few DAC cables, and a $250 switch... you can be well on your way to a 10Gbit network.

memetomancer 5y ago

So many of these posts gainsaying the practice of a home lab are spot on. In my view it's plain foolish to try and cram enterprise gear into a living space. It's almost too hot and loud for my office, why would I take any of that home? The other posts talking about Tiny Mini Micros are on the right track but I think it goes further yet - there's good reason to have a small rack of crap in the corner: - ISP hardware. - pfSense gateway. - Wifi base station. - A good gigabit switch for the house. - Those tiny mini micros or Mac minis for lab stuff. - a NAS chassis or two. - Raspberry Pi clusters. - PiDP-11 or other such hobby stuff that needs a place to sit and blink. There are plenty of other uses too, like security DVRs, ingest stations for cameras/recorders, optical and tape media devices, etc. None of that stuff is hot or loud, but you probably wouldn't want it piled up on your desk or spilling out of some bookshelf. And I think the article kinda gets at that point, tbh.

ChuckMcM 5y ago

I strongly endorse this notion of equipping your own laboratory for your experiments. Learning through doing is always more durable than learning through reading only. While the author is looking at learning about and perfecting their skills as an administrator of networked computer systems there are other "kinds" of laboratories that people set up. Mine, and one I'm more familiar with, are electronics labs. If you're going to be learning about circuits and such it helps to have the basic kit at hand. Similarly for people doing robotics, having a 3D printer in their home lab is essential these days. Nearly everything you might do in a home laboratory will involve some sort of data processing so the ideas by the author are great for creating the lab's "IT infrastructure." In California it also makes it easier to defend the "I built this technology my own gear (picture/description of lab) so you don't own it." But that may be unique to California.

flapjaxy 5y ago
Exactly, I think this thread is focusing on the utility of an environment from a mature SDec POV. Personally having a cheap R710 allowed me to practice my sysadmin, networking, security and a host of other skills that directly translated to my (then) current and future roles. I think the perception of a virtualized environment being the same is false and diminishes the sys/network/security admin's knowledge set. It's is akin to saying that writing JS is the same as Java, coding is coding. I'll also say the number of times I've been in rooms of IT pros who don't understand underlying systems, OS, or networking is disturbingly high. Lastly, even if it is frivolous I see tinkering with hardware as part of the hacking spirit. It's fun, I like it and it gives me something to grumble about with the grey beards.
- ChuckMcM 5y ago
  Here in the Bay Area I know a guy who both wires and de-wires commercial buildings for networking. It is surprising to me how many businesses move out and simply discard any equipment they installed in the POE to support their network. He gave me a Catalyst 9000 series to play with (he usually sells them to the resellers) and it was cool to be playing with "real" hardware but man, that thing was LOUD with the fans.
5y ago

User23 5y ago

UPS plus a generator is a must have if you live someplace with severe weather.