3 ms·
What kind of useful context window are you getting on a 4090, out of curiosity?
by Leynos 5mo ago
What kind of useful context window are you getting on a 4090, out of curiosity?
- crystal_revenge 5mo ago256k tokens for both the MBP and the 4090
- thiisguy 5mo agoCan you share more about your setup please? Like which models and other specs on the machines? Edit: ah I see the models mentioned in another comment of yours