3 ms·
I'm not familiar with all the varied uses of GPUs but it seems like image generation could feasibly be distributed: large upfront download of models, then small
by xnx 3y ago
I'm not familiar with all the varied uses of GPUs but it seems like image generation could feasibly be distributed: large upfront download of models, then small inputs of text and settings, and small output of resulting images.
- WanderPanda 3y agoFor inference I agree! But training requires centralized gradient steps
- vajrabum 3y agoIf you're in a data center and running large training jobs then RDMA over Nvidia Mellanox Infiniband cards over high speed ethernet (like 100GB) are used to ship coefficients around without having that transfer bottleneck in the CPU.
- xxpor 3y ago100 gig, that's considered cute nowadays. https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/ https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances... 3.2 terabits.