3 ms·
Can you recommend any primers about running small models on-device, training/fine tuning on particular datasets (e.g. a specific code base), etc?
by pc86 3y ago
Can you recommend any primers about running small models on-device, training/fine tuning on particular datasets (e.g. a specific code base), etc?
- erichocean 3y agohttps://github.com/ml-explore/mlx https://github.com/ml-explore/mlx https://github.com/ml-explore/mlx-examples https://github.com/ml-explore/mlx-examples This is where all of the action is at right now. You can also use LM Studio: https://lmstudio.ai/ https://lmstudio.ai/
- ParetoOptimal 3y agoAn open source lmstudio software released recently, but I'm having trouble finding a link.
- pmx 3y agoyou might be thinking of GPT4ALL? https://gpt4all.io/index.html https://gpt4all.io/index.html I struggled to get it working on my machine, it kept corrupting the models as it downloaded them.
- dchuk 3y agoWas it this? I haven’t tried it yet but it does look nice. https://jan.ai/ https://jan.ai/
- ParetoOptimal 3y agoYes! That was it.
- rgbrgb 3y agoI maintain this one for native macOS chat https://github.com/psugihara/FreeChat https://github.com/psugihara/FreeChat
- mark_l_watson 3y agoI tried everything for running small models locally, and finally settled on ollama.ai - good command line and REST API functionality. If you like Lisp, recent updates to my Racket and Common Lisp books have examples using the Ollama REST APIs (Can be read online).