3 ms·
As do features. Even if llama.cpp was ~30% slower and the same size as ONNX, something like a custom grammar implementation or an extended context would be a h
by brucethemoose2 3y ago
As do features.
Even if llama.cpp was ~30% slower and the same size as ONNX, something like a custom grammar implementation or an extended context would be a huge deal.
- kanwisher 3y agoONNX doesn’t give you any benefits over something like llama.cpp and it’s significantly slower for zero advantage