4 ms·
Nobody in their right mind is using GCE for training. Take a look at real prices: https://vast.ai/ https://vast.ai/
by execveat 3y ago
Nobody in their right mind is using GCE for training. Take a look at real prices: https://vast.ai/ https://vast.ai/
- bravura 3y agoThese nodes typically have slow downstream, and thus are hard to use when training requires pulling a huge dataset.
- simonw 3y agoI got the impression that kind of thing (buying time on GPUs hosted in people's homes) isn't useful for training large models, because model training requires extremely high bandwidth connections between the GPUs such that you effectively need them in the same rack.
- p1esk 3y agoI suspect most A100s on vast.ai are actually in a datacenter, and might even be on other public clouds, such as AWS. I don't see why either vast.ai or AWS care if this was the case.
- marshray 3y agoIs there at good resource that describes the impact of bandwidth and latency between GPUs? I assume that it's completely impractical to train on distributed systems?
- superpope99 3y agoAren't they explicitly using TPUs in their training? Vast AI are only offering GPUs.
- qeternity 3y agoAnyone training this size of model is almost certainly using AWS/GCE. The GPU marketplaces are nice for people who need smaller/single GPU setups, don't have huge reliability or SLA concerns, and where data privacy risks aren't an issue.
- mrtranscendence 3y agoWell, or Azure.
- qeternity 3y agoHa yes of course. But actually has anyone been able to get instances on Azure? Thought OpenAI had them all reserved.
- jeron 3y agoalso, https://brev.dev/ https://brev.dev/
- lostmsu 3y agoOnly 19 GPUs with 30+G of VRAM in the entire North America. I might be misreading it. It might be just 12 GPUs.