2 ms·
Nice, a rust tool wrapping llama.cpp how does it differ from llama-server? and from llama-swap?
by homarp 1y ago
Nice, a rust tool wrapping llama.cpp
how does it differ from llama-server?
and from llama-swap?
- MKuykendall 1y agoShimmy is designed to be "invisible infrastructure" - the simplest possible way to get local inference working with your existing AI tools. llama-server gives you more control, llama-swap gives you multi-model management. Key differences: - Architecture: llama-swap = proxy + multiple servers, Shimmy = single server - Resource usage: llama-swap runs multiple processes, Shimmy = one 50MB process - Use case: llama-swap for managing many models, Shimmy for simplicity
- MKuykendall 1y agoShimmy is for when you want the absolute minimum footprint - CI/CD pipelines, quick local testing, or systems where you can't install 680MB of dependencies.