MKuykendall

5 ms·

31.
by MKuykendall 1y ago
To use Shimmy (instead of Ollama): 1. Install Shimmy: cargo install shimmy 2. Get GGUF models (same models you'd use with Ollama): # Download to ./models/ directory huggingface-cli download microsoft/Phi-3-mi
32.
by MKuykendall 1y ago
Try cargo install or intentionally exclude, unsigned Rust binaries will do this.
33.
by MKuykendall 1y ago
This should be fixed now!
34.
by MKuykendall 1y ago
Shimmy is for when you want the absolute minimum footprint - CI/CD pipelines, quick local testing, or systems where you can't install 680MB of dependencies.
35.
by MKuykendall 1y ago
Shimmy is designed to be "invisible infrastructure" - the simplest possible way to get local inference working with your existing AI tools. llama-server gives you more control, llama-swap gives you multi-model management. Key
36.
by MKuykendall 1y ago
Hey HN! I built this because I was tired of waiting 10 seconds for Ollama's 680MB binary to start just to run a 4GB model locally. Quick demo - working VSCode + local AI in 30 seconds: curl -L https://github.com/Mich
37.
Show HN: Shimmy – 5MB privacy-first, local alternative to Ollama (680MB)(github.com)
14 points by MKuykendall 1y ago|13 comments
38.
RAG is solving the wrong problem(contextlite.com)
1 points by MKuykendall 1y ago|1 comments
39.
by MKuykendall 1y ago
After watching developers struggle with 200ms+ vector database queries for context retrieval, we realized RAG was fundamentally backwards. Why compute expensive embeddings when you can find context in 0.3ms with SMT-powered reasoning? Conte