3 ms·
Ya but how long will it be before I have 400GB of VRAM on a single GPU? Until then, I can't run these models at home even if the weights were published.
by Der_Einzige 4y ago
Ya but how long will it be before I have 400GB of VRAM on a single GPU? Until then, I can't run these models at home even if the weights were published.
- seydor 4y agothere is already work to reduce the size of these LLMs and it can only get better from now on
- Der_Einzige 4y agoQuantization and pruning have dramatic impacts on model quality, no matter how often they say that it doesn't in papers.