2 ms·It would also help if I had more VRAM and wasn't running a 7B parameter 4-bit quantized model.by fragmede 3y agoIt would also help if I had more VRAM and wasn't running a 7B parameter 4-bit quantized model.