3 ms·

Why would weights need to be « loaded again » for the 2nd pass? Weights never change at inference time no?

• laruss5 23d ago
  [flagged]
• andy12_ 23d ago
  They need to be loaded into shared memory. The weights might fit in global memory if the VRAM is big enough, but they still need to be moved to shared memory for computation.