2 ms·
I've run it with a large context window on 256GB ram + 4090. It wasn't super fast, but it was manageable and it completed the tasks I gave it well.
by kybernetikos 2mo ago
I've run it with a large context window on 256GB ram + 4090. It wasn't super fast, but it was manageable and it completed the tasks I gave it well.
- xiconfjs 2mo agoHow many tps and size of ctx?