3 ms·
Ahhh no sorry I forgot that the actual code controlling this is inside llama-model.cpp ; sorry for the misinfo, the -ngl only set to max by default if you're us
by ngxson 1y ago
Ahhh no sorry I forgot that the actual code controlling this is inside llama-model.cpp ; sorry for the misinfo, the -ngl only set to max by default if you're using Metal backend
(See the code in side llama_model_default_params())
- danielhanchen 1y agoOh no worries! I re-edited my comment to account for it :)