2 ms·
You can, but the reported speeds for such big models were in the range from a token per a dozen seconds to slightly better than 1 or 2 tokens per second.
by adrian_b 3mo ago
You can, but the reported speeds for such big models were in the range from a token per a dozen seconds to slightly better than 1 or 2 tokens per second.