7 ms·

About time. I've noticed ECC DDR3 prices are falling quite a bit in the used market, which is great for my homelab.

nomel 8y ago
Does it make sense to run 128GB of non-ECC? DDRAM is something like 1 bit error per 40 hours per gig [1]. At 128GB, you have 537k of corruption after a week of uptime, assuming you're utilizing it all. [1] https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
TheCowboy 8y ago
It's hard to go wrong for price/performance when building a server out of off-lease enterprise gear, especially with ECC DDR3 being dirt cheap. People might think this means loud rackmount servers, but you can find boards for most form factors. There's a small scene for people who do this and share builds, so it doesn't take much research to get started.
- walterbell 8y ago
  > There's a small scene for people who do this and share builds, so it doesn't take much research to get started. Recommendations on sites/forums?
  - fivefive55 8y ago
    I just found this subreddit yesterday, seems right up your alley. https://www.reddit.com/r/homelab/ https://www.reddit.com/r/homelab/
  - TheCowboy 8y ago
    https://www.serverbuilds.net/ https://www.serverbuilds.net/ which is an offshoot of https://www.reddit.com/user/JDM_WAAAT https://www.reddit.com/user/JDM_WAAAT They also have a Discord. r/homelabs is also good, but it can be more rackmount focused as far as gear recommendations.
copperx 8y ago
Why do you need RAM for chemistry experiments?
lozaning 8y ago
Im running an old opencompute windmill server I picked up when facebook surplussed all of theirs, cant recommend the thing enough. 2 nodes with dual 8 core xeons with HT each, 10GB networking and I've stuffed them with ~256GB of ram, pcie based nvme ssd, and RX480s. Use one as a vm/container host and the other as a workstation. Its more like a freight train than a sports car, but i routinely see Chrome taking up 57 GB of ram on the node I use as my workstation and kinda just laugh.
- akhilcacharya 8y ago
  How much was it? What’s the powe consumption like?
  - lozaning 8y ago
    Looks like ESISO still has some for sale for $381.00[1]. You'll still have to get storage, more memory (if you want), and a gpu. The single PSU of the chassis that supplies power to both nodes does require 220V power, but I solved this easily enough with a 2000 watt step up transformer off of amazon. I've also jerry rigged up an additional ATX power supply as the native chassis psu doesn't have 6 pin power connectors to power the GPUs. I've never calculated how much power the thing pulls under full load, but considering that I've never had any instability I've got to assume it's less than the 2000 watts tha transformer can handle + the 400 watt ATX psu for the GPUs. I've got flat rate electric from my landlord, so it's essentially free to run for me. [1] https://www.ebay.com/itm/QUANTA-OPEN-COMPUTE-SERVER-2x-NODES-4x-8-CORE-E5-2650-2-0GHz-16GB-RAM-NO-HDD/282816028370?hash=item41d9261ed2:g:4FkAAOSwMv5bbxQN https://www.ebay.com/itm/QUANTA-OPEN-COMPUTE-SERVER-2x-NODES...
    - bzillins 8y ago
      The newer winterfell nodes [1] have made their way to ebay at sub 100$ and do not have the exotic power requirements of the earlier dual sled windmill nodes. You can easily power them off of a converted server power supply via diy [2] or commercial [3] means. Windmill nodes have more space on the pcie riser and quieter fans going for them as well. 1) https://www.ebay.com/itm/Quanta-Winterfell-Barebone-Node-2x-HeatSinks-E5-2600-v1-v2-CX341A-2x-PCI-E/202320085031 https://www.ebay.com/itm/Quanta-Winterfell-Barebone-Node-2x-... 2) http://colintd.blogspot.com/2016/10/hacking-hp-common-slot-power-supplies.html http://colintd.blogspot.com/2016/10/hacking-hp-common-slot-p... 3) https://www.parallelminer.com/product/breakout-board-adapter-compatible-with-hp-1200-watt-dps-1200fb/ https://www.parallelminer.com/product/breakout-board-adapter...

Does it make sense to run 128GB of non-ECC? DDRAM is something like 1 bit error per 40 hours per gig [1]. At 128GB, you have 537k of corruption after a week of uptime, assuming you're utilizing it all. [1] https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf

TheCowboy 8y ago

It's hard to go wrong for price/performance when building a server out of off-lease enterprise gear, especially with ECC DDR3 being dirt cheap. People might think this means loud rackmount servers, but you can find boards for most form factors. There's a small scene for people who do this and share builds, so it doesn't take much research to get started.

walterbell 8y ago
> There's a small scene for people who do this and share builds, so it doesn't take much research to get started. Recommendations on sites/forums?
- fivefive55 8y ago
  I just found this subreddit yesterday, seems right up your alley. https://www.reddit.com/r/homelab/ https://www.reddit.com/r/homelab/
- TheCowboy 8y ago
  https://www.serverbuilds.net/ https://www.serverbuilds.net/ which is an offshoot of https://www.reddit.com/user/JDM_WAAAT https://www.reddit.com/user/JDM_WAAAT They also have a Discord. r/homelabs is also good, but it can be more rackmount focused as far as gear recommendations.

copperx 8y ago

Why do you need RAM for chemistry experiments?

lozaning 8y ago

Im running an old opencompute windmill server I picked up when facebook surplussed all of theirs, cant recommend the thing enough. 2 nodes with dual 8 core xeons with HT each, 10GB networking and I've stuffed them with ~256GB of ram, pcie based nvme ssd, and RX480s. Use one as a vm/container host and the other as a workstation. Its more like a freight train than a sports car, but i routinely see Chrome taking up 57 GB of ram on the node I use as my workstation and kinda just laugh.

akhilcacharya 8y ago
How much was it? What’s the powe consumption like?
- lozaning 8y ago
  Looks like ESISO still has some for sale for $381.00[1]. You'll still have to get storage, more memory (if you want), and a gpu. The single PSU of the chassis that supplies power to both nodes does require 220V power, but I solved this easily enough with a 2000 watt step up transformer off of amazon. I've also jerry rigged up an additional ATX power supply as the native chassis psu doesn't have 6 pin power connectors to power the GPUs. I've never calculated how much power the thing pulls under full load, but considering that I've never had any instability I've got to assume it's less than the 2000 watts tha transformer can handle + the 400 watt ATX psu for the GPUs. I've got flat rate electric from my landlord, so it's essentially free to run for me. [1] https://www.ebay.com/itm/QUANTA-OPEN-COMPUTE-SERVER-2x-NODES-4x-8-CORE-E5-2650-2-0GHz-16GB-RAM-NO-HDD/282816028370?hash=item41d9261ed2:g:4FkAAOSwMv5bbxQN https://www.ebay.com/itm/QUANTA-OPEN-COMPUTE-SERVER-2x-NODES...
  - bzillins 8y ago
    The newer winterfell nodes [1] have made their way to ebay at sub 100$ and do not have the exotic power requirements of the earlier dual sled windmill nodes. You can easily power them off of a converted server power supply via diy [2] or commercial [3] means. Windmill nodes have more space on the pcie riser and quieter fans going for them as well. 1) https://www.ebay.com/itm/Quanta-Winterfell-Barebone-Node-2x-HeatSinks-E5-2600-v1-v2-CX341A-2x-PCI-E/202320085031 https://www.ebay.com/itm/Quanta-Winterfell-Barebone-Node-2x-... 2) http://colintd.blogspot.com/2016/10/hacking-hp-common-slot-power-supplies.html http://colintd.blogspot.com/2016/10/hacking-hp-common-slot-p... 3) https://www.parallelminer.com/product/breakout-board-adapter-compatible-with-hp-1200-watt-dps-1200fb/ https://www.parallelminer.com/product/breakout-board-adapter...