4 ms·
Could you recommend a tiny language model I could try out locally?
by politelemon 2y ago
Could you recommend a tiny language model I could try out locally?
- mettamage 2y agoLlama 3.2 has about 3.2b parameters. I have to admit, I use bigger ones like phi-4 (14.7b) and Llama 3.3 (70.6b) but I think Llama 3.2 could do de-anonimization and anonimization of code
- OxfordOutlander 2y ago+1 this idea. I do the same. Just do it locally using ollama, also using 3.2 3b
- RicoElectrico 2y agoLlama 3.2 punches way above its weight. For general "language manipulation" tasks it's good enough - and it can be used on a CPU with acceptable speed.
- seunosewa 2y agoHow many tokens/s?
- iamnotagenius 2y ago10-15t/s on 12400 with ddr5