4 ms·

I'm using open-webui at home with a couple of different models. gemma2-9b fits in VRAM on a NV 3060 card + performs nicely.

• mdp2021 2y ago
  > performs nicely Do you have rough indication of token/s ?
• zakki 2y ago
  What is the memory of your NV3060? 8GB?
  • ngvjmfgb 2y ago
    12GB (edit: that is what mine is)