4 ms·

Might be overloaded – if you have a GPU you can try running it locally by getting the model weights here: https://huggingface.co/bigcode/santacoder https://hugg

by moyix 4y ago

Might be overloaded – if you have a GPU you can try running it locally by getting the model weights here: https://huggingface.co/bigcode/santacoder https://huggingface.co/bigcode/santacoder

bogwog 4y ago
Any idea how much GPU memory you'd need to run this locally? EDIT: just tried it and it didn't seem to go past ~6gb
- lossolo 4y ago
  It's 1 billion model with Fp16 precision so 4-6 GB max.