4 ms·

I am having trouble getting the demo to run. It just errors out

by snowder 4y ago

seinecle 4y ago
Same here
- moyix 4y ago
  Might be overloaded – if you have a GPU you can try running it locally by getting the model weights here: https://huggingface.co/bigcode/santacoder https://huggingface.co/bigcode/santacoder
  - bogwog 4y ago
    Any idea how much GPU memory you'd need to run this locally? EDIT: just tried it and it didn't seem to go past ~6gb
    - lossolo 4y ago
      It's 1 billion model with Fp16 precision so 4-6 GB max.
osanseviero 4y ago
The demo is up again!
enum 4y ago
Give this notebook a shot: https://github.com/arjunguha/BigCode-demos/blob/main/bigcode.ipynb https://github.com/arjunguha/BigCode-demos/blob/main/bigcode... A GPU will help, but I found it passable on a CPU as well.