6 ms·
Hey ilmoi, Do you support multi-node training? Particularly, can I reserve for example a 64-GPU instance via 8 nodes and perform distributed training?
by typon 6y ago
Hey ilmoi,
Do you support multi-node training? Particularly, can I reserve for example a 64-GPU instance via 8 nodes and perform distributed training?
- Proven 6y agoNot much on high speed networking in the answer, which sounds suspicious (it's the most important part)
- ilmoi 6y agoYou can! All machines within a single account are inside a private VLAN, which means they all talk to each other (out of the box, no setup required), but nobody else on gpu.land sees them. See the "Are instances within my account connected?" item in the FAQ - https://gpu.land/faq https://gpu.land/faq If you're going to need 64 GPUs I'll have to increase your account limit (currently 16 GPUs per account). Email me at hi@gpu.land