3 ms·

And vmem use?

• genpfault 1y ago
  ~18.6 GiB, according to nvtop. ollama 0.6.6 invoked with: # server OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve # client ollama run --verbose qwen3:30b-a3b ~19.8 GiB with: /set parameter num_ctx 32768
  • tgtweak 1y ago
    Very nice, should run nicely on a 3090 as well. TY for this. update: wow, it's quite fast - 70-80t/s on LM Studio with a few other applications using GPU.