3 ms·
vast.ai does denote GPU's as either PCIe or SXM (SXM for the A100's and now H100's up there). The SXM bandwidth between GPU's is the full n-way NV-link, the be
by mk_stjames 3y ago
vast.ai does denote GPU's as either PCIe or SXM (SXM for the A100's and now H100's up there). The SXM bandwidth between GPU's is the full n-way NV-link, the best you can get;
They also have a little stat that lists 'per-GPU' bandwidth in GB/s and what PCIe version and speed is being used. So they must run some tests beforehand to gauge this. When I look on there now it varys setup to setup. I see people running quad 4090's on PCIe4.0 x16 with 24GB/s bandwidth between them, some people running x8, some on PCIe3.0 with 11 GB/s, even someone with quad 3090's but all on PCIe 2.0 x1 slots with the bandwidth reading 0.3GB/s!!! (likely an old mining rig with those x1 slots)
- jacquesm 3y agoDo they get more money per instance or is that simply a matter of seeing more utilization?
- mk_stjames 3y agoI've never rented on there so I'm not sure. It seems like the prices are set by the sellers but with some guidance from their performance tool, likely? The have a overall "DLperf" deep learning performance score which is an performance metric and it seems like if you want to get users you set your price per hour at something competitive in line with the market given that metric. For instance, the guy with the quad 3090s on x1 slots actually has an hourly price set HIGHER than everyone else... even with horrible DLPerf score. No one is using that, ever.
- jacquesm 3y agoInteresting... I should try some workloads to see what it can do in practice. Beats having a whole bunch of fans in my room and it seems to be cheaper than the other cloud GPU providers.