4 ms·
I've been running these models on my own laptop without the GPU. They're generally a lot less resource demanding than full LLMs (though thanks to llama.cpp I've
by simonw 3y ago
I've been running these models on my own laptop without the GPU. They're generally a lot less resource demanding than full LLMs (though thanks to llama.cpp I've been able to run those on the CPU as well).
More about the tooling I use to run embedding models locally here: https://simonwillison.net/2023/Sep/4/llm-embeddings/ https://simonwillison.net/2023/Sep/4/llm-embeddings/