3 ms·

Even when there are enterprise SSDs that are mechanically and electrically compatible with the bays/slots in a laptop, they're often untenable due to the lack o

Even when there are enterprise SSDs that are mechanically and electrically compatible with the bays/slots in a laptop, they're often untenable due to the lack of idle power management features on enterprise SSDs (because sleeping when idle kills the latency metrics). Consider that the 480GB DC1000B is officially rated for 1.9W at idle and in practice I wasn't able to get it below 1.2W. Consumer SSDs with working power management would idle under 10mW.

justinclift 2y ago
Interesting, haven't seen mention of that aspect of enterprise ssds at all.
- timschmidt 2y ago
  I usually have to tweak my home rackmount equipment over IPMI to set the fan settings to idle when temps are low instead of 24/7/365 jet engine mode which the local university datacenter seems to prefer. I imagine running the fans full blast all the time results in slightly more reliable performance and reliability, but I don't have the luxury of a soundproof HVAC controlled room to lock everything away in, so the lower noise at lower utilization matters to me. This trade-off definitely happens all over the place in enterprise gear.
  - justinclift 2y ago
    You've been through all the stuff like setting max device power for the NVMe drives and similar yeah? https://nvmexpress.org/resource/technology-power-features/ https://nvmexpress.org/resource/technology-power-features/
    - timschmidt 2y ago
      I just upgraded my main server to an https://www.supermicro.com/products/archive/motherboard/x9drw-3f https://www.supermicro.com/products/archive/motherboard/x9dr... with dual Xeon E5-2680v2s. $15 for 20c/40t and 256gb ECC from the local university surplus. Unfortunately, it's still a little too old to offer NVME. That doesn't matter too much to me, since it's connected via 6gbps SAS to several 24 bay NetApp disk shelves (also $15 ea!) full of disks. I have dealt with PCIe ASPM (https://en.wikipedia.org/wiki/Active_State_Power_Management https://en.wikipedia.org/wiki/Active_State_Power_Management) which similarly trades off between performance, power consumption, and latency. I am not hugely concerned with power consumption as I have solar panels. But noise and excess heat matter to me. My whole stack is consuming around 650W currently. Wildly removed from the days I was running off a creatively overprovisioned OpenWRT wireless router using < 20W. But I'm happy to have the increased capacity for my use cases. Gigabit synchronous fiber to the home is a dangerous drug :)
      - justsomehnguy 2y ago
        Just use PCI-E converter for NVMe. You wouldn't be able to boot from it, but it would work just fine.
        timschmidt 2y ago
        A good suggestion, but it's a 1U server with both available PCIe slots populated between the external SAS controller and a dual 10gbE CNA. I boot from SATA SSDs currently. Could upgrade to enterprise SAS SSDs, but haven't noticed a need for my current uses. The boot drive is rarely touched, as most data terminates at the NetApp disk shelves, which have been a far more reliable way to connect unholy amounts of storage to a machine than any of the consumer direct attached storage options I've tried in the past. There's a mix of SSDs and spinning rust in the disk shelves - SSDs for web server stuff and spinning rust for low cost bulk storage.
        justinclift 2y ago
        > with both available PCIe slots populated between the external SAS controller and a dual 10gbE CNA. Interestingly, there do appear to be PCIe cards that have both external SAS and high speed networking in one: https://forums.servethehome.com/index.php?threads/unique-oem-dual-lsi-2308-mellanox-connectx-3-vpi-combo-expansion-cards.30532/ https://forums.servethehome.com/index.php?threads/unique-oem... Looks like a bunch of mucking around is needed to get things setup, but seems like it works ok from that point onwards: • https://www.ebay.com/itm/314150370285 https://www.ebay.com/itm/314150370285 Have been considering grabbing one to try in a HP Microserver Gen 8 that runs Proxmox. Not a lot of free time at the moment though. ;)
        timschmidt 2y ago
        Neat! And affordable! Thanks! I will definitely look further into them. I got the 10gbE CNA from another university surplus find where a half dozen were being used as 16gb fibre channel adapters with SFP fibre modules. A little digging revealed that they could be switched to 10gbE CNA mode which is a lot more useful for my purposes. Enterprise gear is full of surprises :D Only potential downside I see is the included pcie switch chip. It's dividing up 8gbps between three devices which could independently consume all of that bandwidth. Probably not a huge issue for my uses, but worth considering if you plan on transferring large amounts of data over network and storage simultaneously.
kjkjadksj 2y ago
Thats hardly untenable. Most computers draw a lot more watts where you wouldn't miss 1.2W imo. Even these "low watt" computers are drawing like 5-7 watts at idle and once you do anything on them its a lot more than that.
- wtallis 2y ago
  You wouldn't miss the 1.2-1.9W on a gaming laptop with a discrete GPU. But for more mainstream systems with just integrated graphics, 5W is on the high end for idle power and even Intel's latest mid-range laptop parts can get down to about 3W if you don't have Windows doing dumb shit in the background constantly. Also, on modern laptops you can't trust that components will be powered off when you close the lid, because Microsoft and Intel have been at war with ACPI S3 suspend for years now. So that 1.2W could easily be the minimum power draw of the SSD any time you haven't hibernated or shut down the system.
  - kjkjadksj 2y ago
    You got a source for the 3w? Is this for the cpu or measured at the wall? Seems low. I believe intel n100 can get to 7w but only after heavy tweaking of c states and only with some mobo. A lot of users more in practice from my forum searching on this cpu last year when i almost bought it. Macbook can go down to 5w but again thats idle with nothing running. I don’t think many people leave a laptop computer idling like they might their workstation.
    - ColonelPhantom 2y ago
      My laptop (a Dell Latitude 5490, i5-8350u) with the screen backlight off and at idle goes down to 2.5w power consumption according to the battery statistics. An SSD idling at 1.9w would nearly double it!
    - kldx 2y ago
      Trusting the Thinkpad spec sheet numbers, an AMD T14s lasts >15hrs in one of the benchmarks on a 57Wh battery. That's less than 4W. Perhaps on Linux and with some tuning, it seems plausible to get to 3W?

justinclift 2y ago

Interesting, haven't seen mention of that aspect of enterprise ssds at all.

timschmidt 2y ago
I usually have to tweak my home rackmount equipment over IPMI to set the fan settings to idle when temps are low instead of 24/7/365 jet engine mode which the local university datacenter seems to prefer. I imagine running the fans full blast all the time results in slightly more reliable performance and reliability, but I don't have the luxury of a soundproof HVAC controlled room to lock everything away in, so the lower noise at lower utilization matters to me. This trade-off definitely happens all over the place in enterprise gear.
- justinclift 2y ago
  You've been through all the stuff like setting max device power for the NVMe drives and similar yeah? https://nvmexpress.org/resource/technology-power-features/ https://nvmexpress.org/resource/technology-power-features/
  - timschmidt 2y ago
    I just upgraded my main server to an https://www.supermicro.com/products/archive/motherboard/x9drw-3f https://www.supermicro.com/products/archive/motherboard/x9dr... with dual Xeon E5-2680v2s. $15 for 20c/40t and 256gb ECC from the local university surplus. Unfortunately, it's still a little too old to offer NVME. That doesn't matter too much to me, since it's connected via 6gbps SAS to several 24 bay NetApp disk shelves (also $15 ea!) full of disks. I have dealt with PCIe ASPM (https://en.wikipedia.org/wiki/Active_State_Power_Management https://en.wikipedia.org/wiki/Active_State_Power_Management) which similarly trades off between performance, power consumption, and latency. I am not hugely concerned with power consumption as I have solar panels. But noise and excess heat matter to me. My whole stack is consuming around 650W currently. Wildly removed from the days I was running off a creatively overprovisioned OpenWRT wireless router using < 20W. But I'm happy to have the increased capacity for my use cases. Gigabit synchronous fiber to the home is a dangerous drug :)
    - justsomehnguy 2y ago
      Just use PCI-E converter for NVMe. You wouldn't be able to boot from it, but it would work just fine.
      - timschmidt 2y ago
        A good suggestion, but it's a 1U server with both available PCIe slots populated between the external SAS controller and a dual 10gbE CNA. I boot from SATA SSDs currently. Could upgrade to enterprise SAS SSDs, but haven't noticed a need for my current uses. The boot drive is rarely touched, as most data terminates at the NetApp disk shelves, which have been a far more reliable way to connect unholy amounts of storage to a machine than any of the consumer direct attached storage options I've tried in the past. There's a mix of SSDs and spinning rust in the disk shelves - SSDs for web server stuff and spinning rust for low cost bulk storage.
        justinclift 2y ago
        > with both available PCIe slots populated between the external SAS controller and a dual 10gbE CNA. Interestingly, there do appear to be PCIe cards that have both external SAS and high speed networking in one: https://forums.servethehome.com/index.php?threads/unique-oem-dual-lsi-2308-mellanox-connectx-3-vpi-combo-expansion-cards.30532/ https://forums.servethehome.com/index.php?threads/unique-oem... Looks like a bunch of mucking around is needed to get things setup, but seems like it works ok from that point onwards: • https://www.ebay.com/itm/314150370285 https://www.ebay.com/itm/314150370285 Have been considering grabbing one to try in a HP Microserver Gen 8 that runs Proxmox. Not a lot of free time at the moment though. ;)
        timschmidt 2y ago
        Neat! And affordable! Thanks! I will definitely look further into them. I got the 10gbE CNA from another university surplus find where a half dozen were being used as 16gb fibre channel adapters with SFP fibre modules. A little digging revealed that they could be switched to 10gbE CNA mode which is a lot more useful for my purposes. Enterprise gear is full of surprises :D Only potential downside I see is the included pcie switch chip. It's dividing up 8gbps between three devices which could independently consume all of that bandwidth. Probably not a huge issue for my uses, but worth considering if you plan on transferring large amounts of data over network and storage simultaneously.

kjkjadksj 2y ago

Thats hardly untenable. Most computers draw a lot more watts where you wouldn't miss 1.2W imo. Even these "low watt" computers are drawing like 5-7 watts at idle and once you do anything on them its a lot more than that.

wtallis 2y ago
You wouldn't miss the 1.2-1.9W on a gaming laptop with a discrete GPU. But for more mainstream systems with just integrated graphics, 5W is on the high end for idle power and even Intel's latest mid-range laptop parts can get down to about 3W if you don't have Windows doing dumb shit in the background constantly. Also, on modern laptops you can't trust that components will be powered off when you close the lid, because Microsoft and Intel have been at war with ACPI S3 suspend for years now. So that 1.2W could easily be the minimum power draw of the SSD any time you haven't hibernated or shut down the system.
- kjkjadksj 2y ago
  You got a source for the 3w? Is this for the cpu or measured at the wall? Seems low. I believe intel n100 can get to 7w but only after heavy tweaking of c states and only with some mobo. A lot of users more in practice from my forum searching on this cpu last year when i almost bought it. Macbook can go down to 5w but again thats idle with nothing running. I don’t think many people leave a laptop computer idling like they might their workstation.
  - ColonelPhantom 2y ago
    My laptop (a Dell Latitude 5490, i5-8350u) with the screen backlight off and at idle goes down to 2.5w power consumption according to the battery statistics. An SSD idling at 1.9w would nearly double it!
  - kldx 2y ago
    Trusting the Thinkpad spec sheet numbers, an AMD T14s lasts >15hrs in one of the benchmarks on a 57Wh battery. That's less than 4W. Perhaps on Linux and with some tuning, it seems plausible to get to 3W?