3 ms·
it seems that BERT can be run on the llama.cpp platform https://github.com/ggerganov/llama.cpp/pull/5423 https://github.com/ggerganov/llama.cpp/pull/5423 so pr
by anentropic 3y ago
it seems that BERT can be run on the llama.cpp platform https://github.com/ggerganov/llama.cpp/pull/5423 https://github.com/ggerganov/llama.cpp/pull/5423
so presumably those models could benefit from the speed ups described in OP article when running on CPU
- jerrygenser 3y agollama.cpp only support BERT architectures for embedding but not with classification heads - although there is a feature request to add that
- anentropic 3y agoah I see, thanks, did not read the PR closely!