4 ms·

Quantum chemistry. There are a number of high accuracy wavefunction based methods that are compute-tractable on desktop machines but very demanding of storage.

Quantum chemistry. There are a number of high accuracy wavefunction based methods that are compute-tractable on desktop machines but very demanding of storage. Even when there's a disk based alternative to RAM storage it's a lot slower when you can't just allocate huge arrays in memory. I could easily use 256 GB, maybe 512, on a machine with 8 physical cores. You can get commercial software that is a bit better tuned for running on "ordinary" desktop hardware, but it's so expensive that you don't really come out ahead on total cost.

hengheng 10y ago
Honestly that application sounds like you should be able to just get that 512G machine. Buffered ECC DDR4 RAM has gotten cheaper over the last year too.
- philipkglass 10y ago
  If I still did research full time I'd probably get a Xeon workstation loaded with RAM for home. But it's just a hobby for me now that my day job is writing non-scientific software, so it doesn't seem like a justifiable expense. Might go up from my current 4 cores and 32 GB to something Zen-based later this year. (Tangentially: I wouldn't bother with ECC RAM for this application unless that were required by the rest of the hardware. Memory errors may slightly affect time to a converged solution, but it's rare that a flipped bit would actually cause a job to converge to a different solution.)
  - boulos 10y ago
    If this is just a hobby why not rent a large VM from us (GCE) or AWS? It sounds like you don't run enough to want a box for anything like 24x7, but that it'd be great to have tons of memory and lots of cores (or a giant SSD). We just announced our Skylakes yesterday ;). Disclosure: I work on Google Cloud,
    - lqdc13 10y ago
      I work on NLP projects and have similar issue with memory. The reason I don't use cloud computing is that you constantly have to worry about moving data around, turning stuff on and off, API changes, data movement costs, product lineup changes (should I store this stuff on S3 or local HD), locally compiling all your stuff again or containerization and associated time overhead. Local large box makes way more sense if you're doing solo projects. And if traveling, it would way more sense to rent a VPS from Hetzner. Flat cost structure and tons of bang for the buck. BTW, I you can get 128 gb LGA2011-v3 motherboards. https://www.newegg.com/Product/Product.aspx?Item=N82E16813157543 https://www.newegg.com/Product/Product.aspx?Item=N82E1681315... and it seems like it works: http://www.pcworld.com/article/2938855/hardcore-hardware-we-stuffed-this-pc-with-128gb-of-cutting-edge-ddr4-ram.html http://www.pcworld.com/article/2938855/hardcore-hardware-we-... even though the spec says max 64gb: https://ark.intel.com/products/82932/Intel-Core-i7-5820K-Processor-15M-Cache-up-to-3_60-GHz https://ark.intel.com/products/82932/Intel-Core-i7-5820K-Pro...
      - dsacco 10y ago
        I can confirm, I run machine learning workloads on a home machine with 128GB of RAM on an X99-E WS/USB 3.1 motherboard. My manual explicitly states that it has 128GB RAM support, as well.
paulmd 10y ago
Forget about desktop processors and buy yourself a double-socket LGA2011v3 board and some engineering-sample Xeons. Then you can address that kind of memory and run ECC. The clocks are a little lower than the retail Xeons, and you do need to pay attention to compatible boards/BIOS versions, but you can get a big Xeon E5 for like $200-300.
- geezerjay 10y ago
  > Forget about desktop processors and buy yourself a double-socket LGA2011v3 board and some engineering-sample Xeons. Then you can address that kind of memory and run ECC. If you follow this very thread, you'll notice that another user already complained that "you need to pony up for a Xeon workstation if you need more than 64 GB."
  - paulmd 10y ago
    "Ponying up" here means, in context, Xeon processors that are several thousand dollars each. In contrast, the grandparent was wishing that he could use a Zen that cost several hundred dollars instead. I was pointing out that there were other processors already on the market that met his requirements: Xeon engineering samples, which most people don't think about when they are talking about the costs of high-end Xeon workstations. (This is partially because they are not "finished" products like retail chips. Clocks are lower, and you need to use motherboards/BIOSs from a specific compatibility list, and they are not something you can source from Intel. Thus, they slip many people's minds because they aren't something you would consider for an actual server deployment - but they are perfect for a workstation with specialty needs that needs to hit a tight budget) So yes, I did actually read the thread. You just didn't grasp enough of the context to make a sensible response and figured it was the perfect time for a superslam "did you even read bro?" gotcha response. Typically this sort of response is not welcome on HN. If you don't have something substantiate to post, please don't post at all.
- philipkglass 10y ago
  Thank you for the suggestion. I am surprised how cheap these can be. It has been years since I assembled a system from parts so I had been looking at complete workstations from vendors, but for that much cost saving it is worth considering parts-assembly.

hengheng 10y ago

Honestly that application sounds like you should be able to just get that 512G machine. Buffered ECC DDR4 RAM has gotten cheaper over the last year too.

philipkglass 10y ago
If I still did research full time I'd probably get a Xeon workstation loaded with RAM for home. But it's just a hobby for me now that my day job is writing non-scientific software, so it doesn't seem like a justifiable expense. Might go up from my current 4 cores and 32 GB to something Zen-based later this year. (Tangentially: I wouldn't bother with ECC RAM for this application unless that were required by the rest of the hardware. Memory errors may slightly affect time to a converged solution, but it's rare that a flipped bit would actually cause a job to converge to a different solution.)
- boulos 10y ago
  If this is just a hobby why not rent a large VM from us (GCE) or AWS? It sounds like you don't run enough to want a box for anything like 24x7, but that it'd be great to have tons of memory and lots of cores (or a giant SSD). We just announced our Skylakes yesterday ;). Disclosure: I work on Google Cloud,
  - lqdc13 10y ago
    I work on NLP projects and have similar issue with memory. The reason I don't use cloud computing is that you constantly have to worry about moving data around, turning stuff on and off, API changes, data movement costs, product lineup changes (should I store this stuff on S3 or local HD), locally compiling all your stuff again or containerization and associated time overhead. Local large box makes way more sense if you're doing solo projects. And if traveling, it would way more sense to rent a VPS from Hetzner. Flat cost structure and tons of bang for the buck. BTW, I you can get 128 gb LGA2011-v3 motherboards. https://www.newegg.com/Product/Product.aspx?Item=N82E16813157543 https://www.newegg.com/Product/Product.aspx?Item=N82E1681315... and it seems like it works: http://www.pcworld.com/article/2938855/hardcore-hardware-we-stuffed-this-pc-with-128gb-of-cutting-edge-ddr4-ram.html http://www.pcworld.com/article/2938855/hardcore-hardware-we-... even though the spec says max 64gb: https://ark.intel.com/products/82932/Intel-Core-i7-5820K-Processor-15M-Cache-up-to-3_60-GHz https://ark.intel.com/products/82932/Intel-Core-i7-5820K-Pro...
    - dsacco 10y ago
      I can confirm, I run machine learning workloads on a home machine with 128GB of RAM on an X99-E WS/USB 3.1 motherboard. My manual explicitly states that it has 128GB RAM support, as well.

paulmd 10y ago

Forget about desktop processors and buy yourself a double-socket LGA2011v3 board and some engineering-sample Xeons. Then you can address that kind of memory and run ECC. The clocks are a little lower than the retail Xeons, and you do need to pay attention to compatible boards/BIOS versions, but you can get a big Xeon E5 for like $200-300.

geezerjay 10y ago
> Forget about desktop processors and buy yourself a double-socket LGA2011v3 board and some engineering-sample Xeons. Then you can address that kind of memory and run ECC. If you follow this very thread, you'll notice that another user already complained that "you need to pony up for a Xeon workstation if you need more than 64 GB."
- paulmd 10y ago
  "Ponying up" here means, in context, Xeon processors that are several thousand dollars each. In contrast, the grandparent was wishing that he could use a Zen that cost several hundred dollars instead. I was pointing out that there were other processors already on the market that met his requirements: Xeon engineering samples, which most people don't think about when they are talking about the costs of high-end Xeon workstations. (This is partially because they are not "finished" products like retail chips. Clocks are lower, and you need to use motherboards/BIOSs from a specific compatibility list, and they are not something you can source from Intel. Thus, they slip many people's minds because they aren't something you would consider for an actual server deployment - but they are perfect for a workstation with specialty needs that needs to hit a tight budget) So yes, I did actually read the thread. You just didn't grasp enough of the context to make a sensible response and figured it was the perfect time for a superslam "did you even read bro?" gotcha response. Typically this sort of response is not welcome on HN. If you don't have something substantiate to post, please don't post at all.
philipkglass 10y ago
Thank you for the suggestion. I am surprised how cheap these can be. It has been years since I assembled a system from parts so I had been looking at complete workstations from vendors, but for that much cost saving it is worth considering parts-assembly.