5 ms·

Someone shared this website with me a while back to find used servers on ebay https://www.labgopher.com/ https://www.labgopher.com/ What do you think of that?

• JoshTriplett 9y ago
  > I am trying to put together a render farm where I need a lot of parallel CPUs to run Unreal Engine Swarm [0] but I just don't have the time or experience to execute on it. My goal is 16 Xeon chips across maybe 8 'boxes' to get to around 64/128 cores. These days, one dual-socket server can give you that, for a pretty reasonable cost compared to the time, expense, and power usage of running a rack full of systems.
  • mentos 9y ago
    Yea I think the cost savings when it comes to cores may not make sense in the long run when it comes to setup/teardown and power usage. If you had to put together a copy of the Amazon X1 instance https://aws.amazon.com/ec2/instance-types/x1/ https://aws.amazon.com/ec2/instance-types/x1/ any recommendation on a mobo/memory/case to use?
• cweagans 9y ago
  You might be better off using EC2 or similar. Do the work to configure everything once and package it as an AMI, then when you need your swarm, spin up as many instance as you need, render whatever needs rendered, and then turn off the instances. Should be reasonably cheap, and it'll let you decide on the speed vs cost tradeoff (i.e. more servers cost more, but will get your work done faster; fewer servers cost less, but will take longer). I don't think a rack full of hardware is a very good choice for this.
  • mentos 9y ago
    I will probably try my luck with one of these X1 instances https://aws.amazon.com/ec2/instance-types/x1/ https://aws.amazon.com/ec2/instance-types/x1/ Only fear is that the time/cost it takes to spool up/down would not let us iterate as fast as ideally possible if we owned the hardware and had it running 24/7 for dedicated just in time use. So rather than rent an X1 instance indefinitely we might just pony up the $ upfront and buy our own. "Each X1 instance is powered by four Intel® Xeon® E7 8880 v3 (codenamed Haswell) processors and offers up to 128 vCPUs." Anyone know if the X1 instance is 1 motherboard with 4 sockets or 2 separate mobos with 2?
    • cweagans 9y ago
      The time to spin up is on the order of seconds. IMO, if your core competency isn't sourcing and buying hardware in order to build render farms, you shouldn't do it. It's just not worth it. AWS brings a lot of value to the table, not the least of which is the fact that you won't have to maintain/fix hardware when you're just trying to render something. If the instance is broken, you can just kill it and spin up a new one. Seriously, this will make your life so much easier. > Anyone know if the X1 instance is 1 motherboard with 4 sockets or 2 separate mobos with 2? This shouldn't matter. It's abstracted away from you by design.
• AlphaSite 9y ago
  Have you looked into one of these guys [1]? They go for ~800USD for 16/32 cores/threads. You can get a good high-end build for ~1500USD. It behaves like a modern system with good power profiles and single threaded and multithreaded performance. Or dual epyc for a single box 64/128 system? https://www.newegg.com/Product/Product.aspx?Item=N82E16819113447 https://www.newegg.com/Product/Product.aspx?Item=N82E1681911...
  • mentos 9y ago
    I had done some very basic research on it and found that the performance for the 'Lightmass' program used in Unreal Engine called Swarm was not ideal https://forums.unrealengine.com/community/off-topic/1388243-ue4-dev-rig-threadripper-1950x-or-i7-8700k https://forums.unrealengine.com/community/off-topic/1388243-... From that thread: "Definitely not the threadripper - there were some benchmarks scattered around in other topics and it's actually slower at compiling and lighting builds than my 7820X, while costing more to build a system with." But not sure if I should trust those opinions from the dev community as the benchmarks aren't too sophisticated. I'll continue doing more research on those thank you! I think dual epyc could be interesting for the cost.
• insanebits 9y ago
  You can have a look at xeon e5-2670 v2 cpus, they're relatively cheap, around $200 and have 8c/16t so in a double cpu configuration you could have 64 cores with only 4 machines.
• willglynn 9y ago
  I put together a few dual E5-2683v3 machines for a colo lab earlier this year. Pricing looks a little less favorable today, but I'd ballpark around $1500 per 28 cores/56 threads before RAM and disk. As for contracting, there's lots of options, but everything depends heavily on your usage model. Let's say your render jobs take the shape of "press button, enqueue 10,000 CPU-minutes of work". How often do you press the button? How valuable are low completion times? How penalizing are high completion times? Are there ever times when there is no work? Cloud computing is a good fit for some problems: you can press a button, run your code on 10,000 CPUs, and finish the job in a minute. Cloud computing is expensive for other problems: in terms of hardware, the $1500 machine above compares favorably versus a cc2.8xlarge instance which costs about $1500/month at on-demand pricing. Server hardware is not the only cost -- you still need space and power and disks and network and time for setup and ongoing maintenance -- but there's definitely a break-even point. PaaS, IaaS, dedicated servers, colocation, and in-house datacenters all make sense for users seeking different tradeoffs. It's difficult to give useful advice without knowing more about the tradeoffs you want to make and the relative values you place on setup time versus run time versus money.