3 ms·
Regarding the original llama2.c as I believe the value proposition is to have simple implementation that can execute the inference locally on wide variety of pl
by atairov 3y ago
Regarding the original llama2.c as I believe the value proposition is to have simple implementation that can execute the inference locally on wide variety of platforms. What if we can execute fine-tuned Llama7B on our phones?
- brucethemoose2 3y ago> What if we can execute fine-tuned Llama7B on our phones? 7B and 13B are already quite performant with mlc-llm (which uses an Apache TVM Vulkan/Metal backend). Llama.cpp has the potential to perform well too. These "single file" implementations are not meant to be optimized or feature rich, I dont think.