3 ms·
Really great work, suggest for a next post: the VRAM requirements estimation calculation for running models locally. Especially with different architecture and
by 3abiton 2y ago
Really great work, suggest for a next post: the VRAM requirements estimation calculation for running models locally. Especially with different architecture and different Quants, it gets always confusing and even online calculators give different answer. I never found a really good deep dive on this yet.