5 ms·
Is llama.cpp's CUDA implementation decent? (e.g. does it use CUTLASS properly or something more low level)
by alecco 2y ago
Is llama.cpp's CUDA implementation decent? (e.g. does it use CUTLASS properly or something more low level)
- regularfry 2y agoThe implementation's here: https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/ggml-cuda https://github.com/ggml-org/llama.cpp/tree/master/ggml/src/g...