2 ms·
> philosophy is to achieve That doesn't answer the question, can anyone without more VRAM than sense actually run it as-is or should we wait until they reach t
by moffkalast 4y ago
> philosophy is to achieve
That doesn't answer the question, can anyone without more VRAM than sense actually run it as-is or should we wait until they reach their allegedly impossible aspirational goal?
The very first line of this sort of post should be the specs required and if the trained model weights are actually available, otherwise it's just straight up clickbait.
- nacs 4y agoThe post says it uses "2GB Or less" of VRAM. A 1B parameter transformer model is on the low/tiny-end of model size these days.
- sp332 4y agoModels are usually trained in fp16, meaning 16 bits = 4 bytes per parameter. So a 1B model would take 4GB to begin with, and can be optimised down from there with sparsification and/or quantization. A 50% reduction in size might be noticeably worse than the original, but still useful for boilerplate or other highly predictable patterns.