4 ms·
I fine tuned LLama-2 on code/comment generation (in python) for around $2 and was able to run it natively on an m1 macbook air. I can totally see smaller fine t
by Nevin1901 3y ago
I fine tuned LLama-2 on code/comment generation (in python) for around $2 and was able to run it natively on an m1 macbook air. I can totally see smaller fine tuned LLM's being used locally on consumer devices in the future. I think people underestimate how cheap and efficient this stuff is.
I've actually built a service which lets you fine tune LLama-2/other llms by uploading a JSON dataset. I'm looking for feedback, the link is https://useftn.com https://useftn.com.
- quickthrower2 3y agoWhat is the cost for a useful run? $2.4/Mb sounds good, but how many megabytes did you need for example to do your fine tuning to your satisfaction.
- xur17 3y agoI've been playing with llama-2, and I've been pleasantly surprised with its ability to process images. So you plan to offer image inputs on your fine tuning service?
- kristianp 3y agoThat sounds interesting. Do you have more info on how llama2 can process images?
- xur17 3y agoI've been using this [0] locally, and this model [1] on replicate. [0] https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp [1] https://replicate.com/yorickvp/llava-13b/versions/6bc1c7bb0d2a34e413301fee8f7cc728d2d4e75bfab186aa995f63292bda92fc https://replicate.com/yorickvp/llava-13b/versions/6bc1c7bb0d...