11 ms·
Network: Aruba 1930 24x GbE /w PoE and 4x SFP+ in the basement; Mikrotik CSS610 8x GbE and 2x SFP+ in the home office. Linked with 10GbE fiber, and a ton of VLA
by archi42 3y ago
Network: Aruba 1930 24x GbE /w PoE and 4x SFP+ in the basement; Mikrotik CSS610 8x GbE and 2x SFP+ in the home office.
Linked with 10GbE fiber, and a ton of VLANs.
Server: Dell Precision Rack 7910 with a single E5-2690v4 (14C/28T), 64GB DDR4 ECC (4x16GB), 4x 3.5" HDDs (a hack) with 30TB net storage in RAID1 and 2x small SSDs (OS on RAID1, and temporary data/caches without RAID). I put in the 2x GbE + 2x SFP+ NIC. It's connected with 10GbE and 1GbE.
My old system was nice, but upgrading meant I had to get an expensive CPU; luckily the Dell fell in my hands. With this one, I could double the core count cheaply (it's a 2S system and the CPU is really cheap) and iirc have 192 or 256GB memory per socket (with the cheap 16GB modules).
The above hardware plus all the PoE stuff (5x UniFi AP, a switch behind the TV, DECT-VoIP Gateway), DSL modem and a few ESP draw 110W.
The server itself is in the 80W ballpark (that's 15€/month worth of electricity). This could be reduced with more modern or less powerful hardware. What I did was replace my old, big array of 12 disks with four much bigger disks. Since OS and cashes are on SSD, the disks can spin down a lot. I only insert one PSU, since the second one adds 17W idle power draw.
Compared to other home servers that can be pushed much lower it's still okay, since it replaces a bunch of services I'd have to pay for & allows much faster transfers to my workstation (for backups etc) than the 32MBit/s DSL. The old server was running out of cores... The only external service I still spend money on is mail.
Server runs baremetal Arch Linux (it's not cattle), and two VMs (qemu): OpnSense and Home Assistant.
Samba serves files from the raid to the network.
Services are partially native, some in podman. There are: cockpit, mealy, foundry, step-ca, scanserv, vaultwarden, mosquito, zigbee2mqtt, pihole, tasmoadmin, uptime kuma, UniFi manager, samba, lancache, heimdall, plex. Plus two custom services.
It also interfaces with the PV using RS485. A 20m USB cable connects it to the zigbee stick on the second floor.
I'm looking forward to adding influxdb/graphana for long term monitoring of our heat pump and BEV power consumptions.
The OpnSense does DHCP, wireguard and local DNS.
First step was to setup cockpit, since I like it to configure IPs and access the VMs. Then OpnSense for inter-VLAN routing and firewall, and to the outside world.
Since I wanted to encrypt the traffic even locally, that was the second step: I have a step-ca that serves certs using ACME. My nginx acts as a reverse proxy for most services and gets certs from the step-ca. The CA is limited to the .lan so it can't be used to intercept other traffic. Also, DHCP puts hosts on .dhcp.lan, so random hosts can't just try to pick any domain name inside the network and get certs for that.
With that done, I looked at services I use or that sounded useful, and spun them up.