3 ms·

I wouldn't be surprised if GCP has GPUs scattered throughout the datacenter. If you happen to want to attach one, it has to find one for you to use - potentiall

I wouldn't be surprised if GCP has GPUs scattered throughout the datacenter. If you happen to want to attach one, it has to find one for you to use - potentially live migrating your instance or someone else's so that it can connect them. It'd explain the massive variability between launch times.

master_crab 4y ago
Yeah that was my thought too when I first read the blurb. It’s neat…but like a lot of things in large scale operations, the devil is in the details. GPU-CPU communications is a low latency high bandwidth operation. Not something you can trivially do over standard TCP. GCP offering something like that without the ability to flawlessly migrate the VM or procure enough “local” GPUs means it’s just vaporware. As a side note, I’m surprised the author didn’t note the amount of ICE’s (insufficient capacity errors) AWS throws whenever you spin up a G type instance. AWS is notorious for offering very few G’s and P’s is certain AZs and regions.
- my123 4y ago
  Fungible is selling a GPU decoupling solution via PCIe encapsulated over Ethernet today, so it can certainly be done. And NVIDIA's vGPU solutions do support live migration of GPUs to another host (in which case the vGPU gets moved too, to a GPU on that target).
dubcee349 4y ago
I doubt it would be setup like that. Compute is usually deployed as part of a large set of servers. The reason for that is different compute workloads require different uplink capacity.You don't need a petabyte of uplink capacity for many GPU loads but you may for compute. Just switching ASICs are much more expensive for 400G+ than 100G. That hasn't even got into the optics, NICs and other things. You don't mix and match compute across the same place in the data center traditionally.
ryukoposting 4y ago
I've only ever used AWS for this stuff. When the author said that you could just "add a GPU" to an existing instance, my first reaction was "wow, that sounds like it would be really complicated behind the scenes."

master_crab 4y ago

Yeah that was my thought too when I first read the blurb. It’s neat…but like a lot of things in large scale operations, the devil is in the details. GPU-CPU communications is a low latency high bandwidth operation. Not something you can trivially do over standard TCP. GCP offering something like that without the ability to flawlessly migrate the VM or procure enough “local” GPUs means it’s just vaporware. As a side note, I’m surprised the author didn’t note the amount of ICE’s (insufficient capacity errors) AWS throws whenever you spin up a G type instance. AWS is notorious for offering very few G’s and P’s is certain AZs and regions.

my123 4y ago
Fungible is selling a GPU decoupling solution via PCIe encapsulated over Ethernet today, so it can certainly be done. And NVIDIA's vGPU solutions do support live migration of GPUs to another host (in which case the vGPU gets moved too, to a GPU on that target).

dubcee349 4y ago

I doubt it would be setup like that. Compute is usually deployed as part of a large set of servers. The reason for that is different compute workloads require different uplink capacity.You don't need a petabyte of uplink capacity for many GPU loads but you may for compute. Just switching ASICs are much more expensive for 400G+ than 100G. That hasn't even got into the optics, NICs and other things. You don't mix and match compute across the same place in the data center traditionally.

ryukoposting 4y ago

I've only ever used AWS for this stuff. When the author said that you could just "add a GPU" to an existing instance, my first reaction was "wow, that sounds like it would be really complicated behind the scenes."