3 ms·

Well during training all GPUs were consuming max ~ 450W

• fzzzy 2y ago
  Thanks, good to know. Perhaps it is different for diffusion; with llms, layers are generally split across gpus, meaning inference has to happen on one gpu before the values can be passed between the layer split.
  • Y_Y 2y ago
    That's only if your model is too big for a single GPU and you're not batching.
    • fzzzy 2y ago
      Yes, that's what I was doing. Thanks for the info.