3 ms·
Show HN: Shoehorn – Quantize any model down to run on your machine
Working on Mac, Linux, and Windows now. I include a simple GUI to find new models and get things built and set up. It is working quite well across a few models for me. The GitHub README and DESIGN.md files go into detail of the how/why and it's working remarkably well so far. https://github.com/notactuallytreyanastasio/shoehorn https://github.com/notactuallytreyanastasio/shoehorn
- mbuchel-hn 1mo agodoes this work similar to airllm? i am wondering how it would handle something like quantizing kimi k3 on a budget of 8 gbs, or is that something you are not attempting to solve yet?
- rhgraysonii 1mo agoYes that is exactly what this does.
- kennywinker 1mo agoCould you explain what happens when you try to shoehorn a 2.4T parameter model into a 24gb m4 mac?
- akshay_akula 1mo agoWondering the same thing but for 48gb M5 Max.
- metalliqaz 1mo agoextreme divergence would be my guess
- hmokiguess 1mo agoReminds me of https://github.com/AlexsJones/llmfit https://github.com/AlexsJones/llmfit
- rhgraysonii 1mo agoLLMFit tells you what can run on something. I built something quite similar to their search into Shoehorn now.
- jaylane 1mo agotried it out but based on the model sizing result i got i got an insufficient memory error when the server started running
- rhgraysonii 1mo agoIf you could post an issue if you still have the error around that would be awesome.
- akshay_akula 1mo agoThis is interesting. I wonder how it could work with something like https://github.com/JustVugg/colibri https://github.com/JustVugg/colibri.
- kelvo_ran 1mo ago[dead]
- jedbrooke 1mo agoI gotta laugh at some of the models it suggests, for example: > AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF you’re telling me you managed to fit Fable 5 into just 4B?
- chompychop 1mo ago[flagged]
- unrented7977 1mo agoDon't make fun of people you think are ignorant, it's a pretty shitty look
- chompychop 1mo ago[flagged]
- metalliqaz 1mo agoWell to be fair here... the title of this post doesn't mention fine tuning, it mentions quantization.
- mannyv 1mo agoFyi that model name to me reads Qwen3 4b params distilled/trained with fable 5
- sscarduzio 1mo agoThe project name is perfect!
- puttycat 1mo agoThis is really impressive. Can you say a bit about the underlying process? I'm guessing this is post-training qantization? Isn't PTQ also resource-intensive? (Ie might not work on any machine)
- vancekai 1mo ago[dead]