3 ms·
Even in tensor parallel modes? I thought it could only work if you're fine stalling all but n GPU for n users at any given moments.
by numpad0 10mo ago
Even in tensor parallel modes? I thought it could only work if you're fine stalling all but n GPU for n users at any given moments.