3 ms·
The performance will probably be similar as long as you remember to tune the settings listed here: https://github.com/ollama/ollama/blob/main/docs/api.md https:
by hellsten 3y ago
The performance will probably be similar as long as you remember to tune the settings listed here: https://github.com/ollama/ollama/blob/main/docs/api.md https://github.com/ollama/ollama/blob/main/docs/api.md
Try to, for example, set 'num_gpu' to 99 and 'use_mlock' to true.