3 ms·
Late edit: Deep Speed, not FlexGen. I don't know if FG could work, but that repo only supports it for Opt models.
by sp332 4y ago
Late edit: Deep Speed, not FlexGen. I don't know if FG could work, but that repo only supports it for Opt models.
loading thread…