4 ms·

Went down a similar route for a 'mini virtualization server.' HP ProDesk 405 G6 with an AMD Ryzen 7 PRO 4750GE (8C/16T, 3.1GHz base clock, 8 VEGA GPU cores), a

by MrFoof 5y ago

Went down a similar route for a 'mini virtualization server.' HP ProDesk 405 G6 with an AMD Ryzen 7 PRO 4750GE (8C/16T, 3.1GHz base clock, 8 VEGA GPU cores), and tossed in 64GB of DDR4-3200 and a 1TB Samsung 980 Pro. Real storage is like 55TB of RAID 1 or RAID 10 arrays on the network, where VM backups and snapshots also go (and get shoved out to Backblaze), as well as media and other chunky data. If you got an EliteDesk 805 you can even have a second M.2 socket so you can RAID1 the SSDs. Only downer is no 10GbE ethernet. Sits at about 12W at idle. With everything under the sun running it hums along at about 20-22W of power consumption. Absolutely silent. Plus being an APU I can even have a Plex VM happily transcode without breaking a sweat. AMD DASH means I also have full IPMI functionality as well (not as good as Intel vPro, but good enough). 1L form factors (incl. Mac Mini, NUCs) are ideal home servers at this point when you need much more power than a Raspberry Pi provides.

Hamuko 5y ago
>1L form factors (incl. Mac Mini, NUCs) are ideal home servers at this point when you need much more power than a Raspberry Pi provides. A Synology or a HPE MicroServer might be better since you can use it for storage as well without external HDDs.
- walterbell 5y ago
  Storage devices are so quirky that I've given up on integration of SFF compute with storage. Compact storage devices make tradeoffs, which may or may not fit your needs, e.g. flash/SSD tier, 10GbE/2.5GbE NICs, memory, TPMs, low-power vs. cores. External storage offers more flexibility, which will increase as USB4/Thunderbolt become widely available and fall in price. For now, USB 3.2 JBOD with ZFS is workable with external enclosures, https://oyendigital.com/multi-bay-storage https://oyendigital.com/multi-bay-storage Wish there was a NUC-sized device with an LSI SATA controller and external connector. If we're lucky, the https://frame.work https://frame.work "modular laptop" approach will be adapted to compact desktops based on laptop chipsets.
- MrFoof 5y ago
  I don't have, "external HDDs". I have a NAS running TrueNAS, with a few arrays in it depending on the performance needed for the storage. Its primary jobs are for data processing and video editing (plus smaller arrays for backups and VM snapshots), and has multiple 10GbE network interfaces (which it can saturate) to facilitate that. It was also built to sip power when there isn't any disk activity. I have no desire for it to run VMs or containers. Its job is to just be storage. The virtualization server is one of many things that connects to it. There are benefits to having the storage and the virtualization servers separate, mostly that there's no single point of failure.
  - gtdawg 5y ago
    What hardware profile runs your true nas? I've been looking to build something open source rather than using Synology.
    - MrFoof 5y ago
      I'm using an AMD EPYC Embedded 3201. This is an 8-core (no hyperthreading) Zen 1-based processor with a 1.5GHz base clock and 3.1GHz max boost and 30W TDP. It came on a SuperMicro M11SDV-8CT-LN4F motherboard to which I've added a dual port 10GbE network adapter. Most of the four 1GbE ports on the board are disabled save for the one supporting IPMI. I'd prefer a newer generation Zen CPU, but AMD doesn't currently have a recent embedded-application CPU with anything approaching that low a TDP. Something like an Intel Atom C3758 would be the Intel comparison point, but the EPYC Embedded uses a bit less power at idle (the base clocks of the C3758 are higher), and the EPYC Embedded's ability to turbo means it has more headroom when required for only about 5W additional power draw. Moreover, the EPYC Embedded has about 170% more overall CPU performance than the Atom.
minimaul 5y ago
There are some nice self-build options too - eg I have https://www.asrock.com/Nettop/Intel/DeskMini%20310%20Series/index.asp https://www.asrock.com/Nettop/Intel/DeskMini%20310%20Series/... with a i7 9700 in (8c8t), plus 1x NVMe and 2x 2TB SATA SSDs for VM disk. Works really really well, and it's pretty much silent. NUCs are just that little bit too compromised, I think - but things like the hp prodesk or the asrock deskmini are a really nice sweet spot.
- pizza234 5y ago
  The power draw of an i7-9700 is around 100W on full load (107W, CPU only, according to Tom's hardware¹). There are relatively quiet solutions, but they're in the ballpark of 30/35 dB, and they may not even cover the 100+W requirement. 22W of full load can be cooled with a passive cooler (large, but still can). [¹] https://www.tomshardware.com/reviews/intel-core-i7-9700k-9th-gen-cpu,5876-2.html https://www.tomshardware.com/reviews/intel-core-i7-9700k-9th...
  - minimaul 5y ago
    I am using a https://noctua.at/en/nh-l9i https://noctua.at/en/nh-l9i as the cooler for it and it keeps it in a sensible range. I don't really max out all of the cores frequently, and certainly not with the most power-hungry workloads - eg AVX2. Noctua only recommend this with the following caveats: Recommended with good case ventilation only. If power limits are disabled in BIOS, CPU might not be able to keep maximum turbo clock under prolonged AVX loads, suggested maximum power limit: 140W For case ventilation - the fan draws air straight from outside of the case onto the heatsink in the DeskMini chassis, so that's covered. My workload is mostly VMs & containers - although it also runs a Gitlab CI build instance. Practically, I've not had any issues. Typical CPU temps are in the 30s or 40s according to coretemp. Obviously, YMMV! edit: for me, this replaced a nuc8i5beh that was very loud under typical workloads. Changed from a laptop-style very whiny fan to the inaudible noise of the noctua, and I got a lot more performance out of it as well.
pengaru 5y ago
> it hums along at about 20-22W of power consumption. > Absolutely silent Well which is it, "absolutely silent" or "humming"? Does this thing have fans?
- ElCapitanMarkla 5y ago
  I think “hums along” here is meant more as “speeds along” vs actually humming.
  - pengaru 5y ago
    Sooo... this machine is fanless and absolutely silent then?
paranoidrobot 5y ago
> Only downer is no 10GbE ethernet. I've been trying to solve this problem in the ~1L chassis size. I have two problems I'm trying to solve: - Multi-port 10GbE for a PFSense/OpnSense router replacement - Reasonable GPU for Plex Transcoding - at least two concurrent 4K HEVC decodes + re-encodes to 4K or 1080p (Intel QuickSync isn't fast enough for this) Both configurations of them have to be silent, or near silent. I think the Lenovo P330 Tiny is probably the best option here, as it has a PCIe Gen 3 x8 slot. The factory options let you put either a quad-port 1GbE NIC in, or a Nvidia Quadro P620/P1000. I think I should be able to put an Intel X710-DA4 (quad-port SFP+ NIC) in for my PFSense box, or upgrade to a Quadro P2000/2200 for the Plex transcode needs.
- geoffeg 5y ago
  I've been looking for something similar, but lower power usage is more important to me than silence.
- MrFoof 5y ago
  For a custom build it's simply difficult to cram all of that in if you want a very small form factor. You may want to look at the DEC840 (AMD EPYC Embedded 3101) or DEC850 (AMD EPYC Embedded 3201) that uses Deciso's Netboard A20 and is sold on the OPNSense website's shop. It has two SFP+ transceivers and four 1GbE ports in a 199x186x46mm form factor. It is entirely passively cooled (the enclosure itself is a giant heatsink) and contains zero fans. They don't yet have this available for sale in a 1U rackmount enclosure, but I assume it's in the works. From what I've heard it tends to use about 15W or so of power at idle and more around 20ish under some real loads. Not bad for something that can supposedly shove 2.3Gbps of IPsec VPN traffic! It won't cover all of your use cases, as it would require a custom build, but if you're looking for a super beefy router (that you can just connect to an aggregation switch) it's an option. You could probably use it for the next 10-15 years and it'd likely still serve all of your needs.
  - paranoidrobot 5y ago
    That DEC840, by the time it lands here in Australia is over AUD$2100 (EUR999, plus Euro VAT, plus Australian GST). While I'm sure it's a really nice unit, it's just too expensive for my home use. The Lenovo P330 Tiny (I7-8700T, 16GB RAM, 256GB m.2 SSD) I can get for AUD$750, and a X710-DA4 for AUD$300 - so roughly half the price, and it'll give me more SFP+ ports. I'll give that having dedicated hardware like that is probably going to be more reliable. I've got a Netgate two-port unit that's been going great for years who's only problem is that it's onboard eMMC storage started going wonky. But hey, if it doesn't work out as an OpnSense router, well I'll just stick an nVidia Quadro in it and make it the Plex server instead.
znpy 5y ago
> AMD DASH means I also have full IPMI functionality as well could you link some resources about this ?
- MrFoof 5y ago
  AMD PRO Manageability: https://www.amd.com/en/technologies/security-manageability https://www.amd.com/en/technologies/security-manageability Tools for DMTF Dash: https://developer.amd.com/tools-for-dmtf-dash/ https://developer.amd.com/tools-for-dmtf-dash/ There are both CLI tools (including for Linux), a Windows AMD Dash Management UI, as well as Microsoft SCCM plugins. The virtual desktop is basically a VNC server. The only real implementer at this point seems to be RealTek, so you only have the AMD Dash functionality I believe through the onboard RealTek NIC. You can enable Dash in the BIOS. You also need to update the DASH firmware of the RealTek network card to enable DASH to truly work. That's software you can get through a OEM (like HP or Lenovo) from their support downloads. I've tried it. I can power off, restart, go into the BIOS, have USB pass-through for installing an OS, and VNC in. It's not anywhere as good as Intel vPro, but it works.

by MrFoof 5y ago

Hamuko 5y ago

>1L form factors (incl. Mac Mini, NUCs) are ideal home servers at this point when you need much more power than a Raspberry Pi provides. A Synology or a HPE MicroServer might be better since you can use it for storage as well without external HDDs.

walterbell 5y ago
Storage devices are so quirky that I've given up on integration of SFF compute with storage. Compact storage devices make tradeoffs, which may or may not fit your needs, e.g. flash/SSD tier, 10GbE/2.5GbE NICs, memory, TPMs, low-power vs. cores. External storage offers more flexibility, which will increase as USB4/Thunderbolt become widely available and fall in price. For now, USB 3.2 JBOD with ZFS is workable with external enclosures, https://oyendigital.com/multi-bay-storage https://oyendigital.com/multi-bay-storage Wish there was a NUC-sized device with an LSI SATA controller and external connector. If we're lucky, the https://frame.work https://frame.work "modular laptop" approach will be adapted to compact desktops based on laptop chipsets.
MrFoof 5y ago
I don't have, "external HDDs". I have a NAS running TrueNAS, with a few arrays in it depending on the performance needed for the storage. Its primary jobs are for data processing and video editing (plus smaller arrays for backups and VM snapshots), and has multiple 10GbE network interfaces (which it can saturate) to facilitate that. It was also built to sip power when there isn't any disk activity. I have no desire for it to run VMs or containers. Its job is to just be storage. The virtualization server is one of many things that connects to it. There are benefits to having the storage and the virtualization servers separate, mostly that there's no single point of failure.
- gtdawg 5y ago
  What hardware profile runs your true nas? I've been looking to build something open source rather than using Synology.
  - MrFoof 5y ago
    I'm using an AMD EPYC Embedded 3201. This is an 8-core (no hyperthreading) Zen 1-based processor with a 1.5GHz base clock and 3.1GHz max boost and 30W TDP. It came on a SuperMicro M11SDV-8CT-LN4F motherboard to which I've added a dual port 10GbE network adapter. Most of the four 1GbE ports on the board are disabled save for the one supporting IPMI. I'd prefer a newer generation Zen CPU, but AMD doesn't currently have a recent embedded-application CPU with anything approaching that low a TDP. Something like an Intel Atom C3758 would be the Intel comparison point, but the EPYC Embedded uses a bit less power at idle (the base clocks of the C3758 are higher), and the EPYC Embedded's ability to turbo means it has more headroom when required for only about 5W additional power draw. Moreover, the EPYC Embedded has about 170% more overall CPU performance than the Atom.

minimaul 5y ago

There are some nice self-build options too - eg I have https://www.asrock.com/Nettop/Intel/DeskMini%20310%20Series/index.asp https://www.asrock.com/Nettop/Intel/DeskMini%20310%20Series/... with a i7 9700 in (8c8t), plus 1x NVMe and 2x 2TB SATA SSDs for VM disk. Works really really well, and it's pretty much silent. NUCs are just that little bit too compromised, I think - but things like the hp prodesk or the asrock deskmini are a really nice sweet spot.

pizza234 5y ago
The power draw of an i7-9700 is around 100W on full load (107W, CPU only, according to Tom's hardware¹). There are relatively quiet solutions, but they're in the ballpark of 30/35 dB, and they may not even cover the 100+W requirement. 22W of full load can be cooled with a passive cooler (large, but still can). [¹] https://www.tomshardware.com/reviews/intel-core-i7-9700k-9th-gen-cpu,5876-2.html https://www.tomshardware.com/reviews/intel-core-i7-9700k-9th...
- minimaul 5y ago
  I am using a https://noctua.at/en/nh-l9i https://noctua.at/en/nh-l9i as the cooler for it and it keeps it in a sensible range. I don't really max out all of the cores frequently, and certainly not with the most power-hungry workloads - eg AVX2. Noctua only recommend this with the following caveats: Recommended with good case ventilation only. If power limits are disabled in BIOS, CPU might not be able to keep maximum turbo clock under prolonged AVX loads, suggested maximum power limit: 140W For case ventilation - the fan draws air straight from outside of the case onto the heatsink in the DeskMini chassis, so that's covered. My workload is mostly VMs & containers - although it also runs a Gitlab CI build instance. Practically, I've not had any issues. Typical CPU temps are in the 30s or 40s according to coretemp. Obviously, YMMV! edit: for me, this replaced a nuc8i5beh that was very loud under typical workloads. Changed from a laptop-style very whiny fan to the inaudible noise of the noctua, and I got a lot more performance out of it as well.

pengaru 5y ago

> it hums along at about 20-22W of power consumption. > Absolutely silent Well which is it, "absolutely silent" or "humming"? Does this thing have fans?

ElCapitanMarkla 5y ago
I think “hums along” here is meant more as “speeds along” vs actually humming.
- pengaru 5y ago
  Sooo... this machine is fanless and absolutely silent then?

paranoidrobot 5y ago

> Only downer is no 10GbE ethernet. I've been trying to solve this problem in the ~1L chassis size. I have two problems I'm trying to solve: - Multi-port 10GbE for a PFSense/OpnSense router replacement - Reasonable GPU for Plex Transcoding - at least two concurrent 4K HEVC decodes + re-encodes to 4K or 1080p (Intel QuickSync isn't fast enough for this) Both configurations of them have to be silent, or near silent. I think the Lenovo P330 Tiny is probably the best option here, as it has a PCIe Gen 3 x8 slot. The factory options let you put either a quad-port 1GbE NIC in, or a Nvidia Quadro P620/P1000. I think I should be able to put an Intel X710-DA4 (quad-port SFP+ NIC) in for my PFSense box, or upgrade to a Quadro P2000/2200 for the Plex transcode needs.

geoffeg 5y ago
I've been looking for something similar, but lower power usage is more important to me than silence.
MrFoof 5y ago
For a custom build it's simply difficult to cram all of that in if you want a very small form factor. You may want to look at the DEC840 (AMD EPYC Embedded 3101) or DEC850 (AMD EPYC Embedded 3201) that uses Deciso's Netboard A20 and is sold on the OPNSense website's shop. It has two SFP+ transceivers and four 1GbE ports in a 199x186x46mm form factor. It is entirely passively cooled (the enclosure itself is a giant heatsink) and contains zero fans. They don't yet have this available for sale in a 1U rackmount enclosure, but I assume it's in the works. From what I've heard it tends to use about 15W or so of power at idle and more around 20ish under some real loads. Not bad for something that can supposedly shove 2.3Gbps of IPsec VPN traffic! It won't cover all of your use cases, as it would require a custom build, but if you're looking for a super beefy router (that you can just connect to an aggregation switch) it's an option. You could probably use it for the next 10-15 years and it'd likely still serve all of your needs.
- paranoidrobot 5y ago
  That DEC840, by the time it lands here in Australia is over AUD$2100 (EUR999, plus Euro VAT, plus Australian GST). While I'm sure it's a really nice unit, it's just too expensive for my home use. The Lenovo P330 Tiny (I7-8700T, 16GB RAM, 256GB m.2 SSD) I can get for AUD$750, and a X710-DA4 for AUD$300 - so roughly half the price, and it'll give me more SFP+ ports. I'll give that having dedicated hardware like that is probably going to be more reliable. I've got a Netgate two-port unit that's been going great for years who's only problem is that it's onboard eMMC storage started going wonky. But hey, if it doesn't work out as an OpnSense router, well I'll just stick an nVidia Quadro in it and make it the Plex server instead.

znpy 5y ago

> AMD DASH means I also have full IPMI functionality as well could you link some resources about this ?

MrFoof 5y ago
AMD PRO Manageability: https://www.amd.com/en/technologies/security-manageability https://www.amd.com/en/technologies/security-manageability Tools for DMTF Dash: https://developer.amd.com/tools-for-dmtf-dash/ https://developer.amd.com/tools-for-dmtf-dash/ There are both CLI tools (including for Linux), a Windows AMD Dash Management UI, as well as Microsoft SCCM plugins. The virtual desktop is basically a VNC server. The only real implementer at this point seems to be RealTek, so you only have the AMD Dash functionality I believe through the onboard RealTek NIC. You can enable Dash in the BIOS. You also need to update the DASH firmware of the RealTek network card to enable DASH to truly work. That's software you can get through a OEM (like HP or Lenovo) from their support downloads. I've tried it. I can power off, restart, go into the BIOS, have USB pass-through for installing an OS, and VNC in. It's not anywhere as good as Intel vPro, but it works.