4 ms·

I'm amazed at how autoencoding actually works, despite the simplicity of the approach. I've dabbled with this stuff quite a bit, but I tend towards stuff I can

I'm amazed at how autoencoding actually works, despite the simplicity of the approach. I've dabbled with this stuff quite a bit, but I tend towards stuff I can run on my own computer, rather than trusting any service to remain stable for years or decades. Are there any good document embedding models I can run on my own hardware? Word2Vec is interesting, but I'd like to also try out cross-linking blog entries, etc.

SushiHippie 3y ago
I've found gpt4all to be quite good and runs locally. https://docs.gpt4all.io/gpt4all_python_embedding.html https://docs.gpt4all.io/gpt4all_python_embedding.html
navbaker 3y ago
The HuggingFace embedding models are really good! See here for a good intro tutorial: https://huggingface.co/blog/getting-started-with-embeddings https://huggingface.co/blog/getting-started-with-embeddings
ahoho 3y ago
sentence-transformers is excellent. It will be slow on CPU but, because it’s on top of pytorch, does have apple silicon/GPU support.
simonw 3y ago
I've been running these models on my own laptop without the GPU. They're generally a lot less resource demanding than full LLMs (though thanks to llama.cpp I've been able to run those on the CPU as well). More about the tooling I use to run embedding models locally here: https://simonwillison.net/2023/Sep/4/llm-embeddings/ https://simonwillison.net/2023/Sep/4/llm-embeddings/
kordlessagain 3y ago
Install Instructor and dependencies: pip install InstructorEmbedding pip install torch pip install numpy pip install tqdm pip install sentence_transformers Use it: from InstructorEmbedding import INSTRUCTOR import numpy as np from scipy.spatial.distance import cosine xl = INSTRUCTOR('hkunlp/instructor-xl') embeddings = xl.encode(["I'm amazed at how autoencoding actually works, despite the simplicity of the approach.", "Are there any good document embedding models I can run on my own hardware? "]).tolist() cosine_distance = 1 - np.dot(embeddings[0], embeddings[1]) / (np.linalg.norm(embeddings[0]) * np.linalg.norm(embeddings[1])) print(cosine_distance) Output: 0.33516267980261427

SushiHippie 3y ago

I've found gpt4all to be quite good and runs locally. https://docs.gpt4all.io/gpt4all_python_embedding.html https://docs.gpt4all.io/gpt4all_python_embedding.html

navbaker 3y ago

The HuggingFace embedding models are really good! See here for a good intro tutorial: https://huggingface.co/blog/getting-started-with-embeddings https://huggingface.co/blog/getting-started-with-embeddings

ahoho 3y ago

sentence-transformers is excellent. It will be slow on CPU but, because it’s on top of pytorch, does have apple silicon/GPU support.

simonw 3y ago

I've been running these models on my own laptop without the GPU. They're generally a lot less resource demanding than full LLMs (though thanks to llama.cpp I've been able to run those on the CPU as well). More about the tooling I use to run embedding models locally here: https://simonwillison.net/2023/Sep/4/llm-embeddings/ https://simonwillison.net/2023/Sep/4/llm-embeddings/

kordlessagain 3y ago

Install Instructor and dependencies: pip install InstructorEmbedding pip install torch pip install numpy pip install tqdm pip install sentence_transformers Use it: from InstructorEmbedding import INSTRUCTOR import numpy as np from scipy.spatial.distance import cosine xl = INSTRUCTOR('hkunlp/instructor-xl') embeddings = xl.encode(["I'm amazed at how autoencoding actually works, despite the simplicity of the approach.", "Are there any good document embedding models I can run on my own hardware? "]).tolist() cosine_distance = 1 - np.dot(embeddings[0], embeddings[1]) / (np.linalg.norm(embeddings[0]) * np.linalg.norm(embeddings[1])) print(cosine_distance) Output: 0.33516267980261427