3 ms·

Are you potentially interested in renting out idle capacity for batch jobs? If so, what kind of interconnect do you have? Feel free to contact me (info in my pr

• qeternity 6y ago
  Sadly, given how cheap the infra is, it's not worth it to us to have to share with someone else. Let's say we could cost share 50% thus saving us $12k/yr...we would spend a lot more than $12k setting up a system and all the headache that arise from sharing the infra. But thanks for the offer! The natural market forces will drive cloud computing prices down the same way they've driven everything else down. But until then, roll-your-own can save loads.
  • namibj 6y ago
    I figured it might be unreasonable, but thanks for responding. Yeah, I was particularly curious because I was unable to find better public offers than AWS (with their homebrew 100Gbit/s MPI that drops infiniband's hardware-guaranteed-delivery to prevent statically-allocated-buffer issues in many-node setups, allowing them quite impressive scalability) or Azure (with their 200Gbit/s Infiniband clusters), at least for occasional batch-jobs. I wouldn't ask if I could DIY for less than using AWS, but owning ram is expensive. And for development purposes it would be quite enticing to just co-locate storage with compute, and rent some space on those NVMe drives for the hours/days you're running e.g. individual iterations on a large dataset to do accurate profile-guided optimizations (by hand or by compiler). Iterations only take a few minutes each, but loading what's essentially a good fraction (minus scratch space, and some compression is typically possible) of the ram over network causes setup to take quite a long time (compared to a single iteration).