2 ms·
> VRAM isn't moving nearly as fast as the models are progressing in size. Models of any given quality are declining in size (both number of parameters, and als
by dragonwriter 3y ago
> VRAM isn't moving nearly as fast as the models are progressing in size.
Models of any given quality are declining in size (both number of parameters, and also VRAM required for inference per parameter because quantization methods are improving.)