4 ms·

> For those who go, "but I need real power" I think those usually also want real storage space, NVMe either can’t offer that, or it would make the build so exp

> For those who go, "but I need real power" I think those usually also want real storage space, NVMe either can’t offer that, or it would make the build so expensive, electricity stops mattering. And in those cases, 4-12 24 TB HDDs will out-consume your other hardware anyway ;)

dusanh 4y ago
It depends how one uses the storage, but on my home server I didn't need to access the (external) HDD storage often so I created a systemd service that spun it down after set time of inactivity. The response when the HDD is needed again is not instant, but okay enough for what I needed. /usr/sbin/hdparm -S 120 /dev/sdb https://wiki.archlinux.org/title/Hdparm https://wiki.archlinux.org/title/Hdparm
MrFoof 4y ago
Yep. I have a storage server, also with a 5750GE, 128GB ECC memory, 40Gbps (QSFP+, passive DAC), EIGHT 16TB drives off a modern HBA, two 2TB NVME SSDs (mirrored) as a cache/ingest tier, a SATA DOM to boot off of, and a BMC (which alone draws 3.2W). 132TB raw storage organized into 16TB and 64TB ZFS volumes, plus the 2TB ingest/cache tier. Can saturate 40Gbps up and down. 43-44W at the lowest idle state, about 55W with just the 16TB volume spinning. Though with all the memory and the cache/ingest tier, it doesn’t spin up often, and often it’s mostly spinning up just the two-disk volume. So most of the time it is completely silent — not even a fan whirr. And that provides the bulk storage and backups (plus pushing offsite) to the PCs, a six-node Pi cluster, and those 9W virtualization servers by providing iSCSI volumes. Also lets any new device PXE boot, and if a device has no OS and no configuration, the hardware will self-install, then self-configure after the first boot. A lot of work, but very valuable experience that paid off at the day job. A lot of VERY deliberate hardware choices to drive power consumption that low, and probably two weeks measuring power consumption of everything. Something people fail to realize that 10GbE Ethernet is rather power hungry if you’re using copper, but if you’re using fiber or (ideally) passive DACs, you can eke out lower latency, better performance, and drive power consumption into the ground. It matters when you consider you pay for that power consumption for 10GbE copper at every single RJ-45 port in the chain between the devices. You can pay < 10W for all the devices in a 40Gbps connection (four ports) with passive DAC, but with copper it’s more like 40-42W.

dusanh 4y ago

It depends how one uses the storage, but on my home server I didn't need to access the (external) HDD storage often so I created a systemd service that spun it down after set time of inactivity. The response when the HDD is needed again is not instant, but okay enough for what I needed. /usr/sbin/hdparm -S 120 /dev/sdb https://wiki.archlinux.org/title/Hdparm https://wiki.archlinux.org/title/Hdparm

MrFoof 4y ago

Yep. I have a storage server, also with a 5750GE, 128GB ECC memory, 40Gbps (QSFP+, passive DAC), EIGHT 16TB drives off a modern HBA, two 2TB NVME SSDs (mirrored) as a cache/ingest tier, a SATA DOM to boot off of, and a BMC (which alone draws 3.2W). 132TB raw storage organized into 16TB and 64TB ZFS volumes, plus the 2TB ingest/cache tier. Can saturate 40Gbps up and down. 43-44W at the lowest idle state, about 55W with just the 16TB volume spinning. Though with all the memory and the cache/ingest tier, it doesn’t spin up often, and often it’s mostly spinning up just the two-disk volume. So most of the time it is completely silent — not even a fan whirr. And that provides the bulk storage and backups (plus pushing offsite) to the PCs, a six-node Pi cluster, and those 9W virtualization servers by providing iSCSI volumes. Also lets any new device PXE boot, and if a device has no OS and no configuration, the hardware will self-install, then self-configure after the first boot. A lot of work, but very valuable experience that paid off at the day job. A lot of VERY deliberate hardware choices to drive power consumption that low, and probably two weeks measuring power consumption of everything. Something people fail to realize that 10GbE Ethernet is rather power hungry if you’re using copper, but if you’re using fiber or (ideally) passive DACs, you can eke out lower latency, better performance, and drive power consumption into the ground. It matters when you consider you pay for that power consumption for 10GbE copper at every single RJ-45 port in the chain between the devices. You can pay < 10W for all the devices in a 40Gbps connection (four ports) with passive DAC, but with copper it’s more like 40-42W.