3 ms·

would appreciate resources on fine tuning

brucethemoose2 3y ago
Honestly, I dunno. I think most people are using lit-llama or EasyLM (on TPUs) for finetuning? QLORA is the gold standard for more affordable training. As for datasets, just look at the open datasets the best-in-class models are using, like Vicuna or https://huggingface.co/NousResearch/Nous-Hermes-13b https://huggingface.co/NousResearch/Nous-Hermes-13b Some model datasets like Manticore, Chronos or the infamous Pygmalion are more "secretive," but you can find the dataset gathering scripts on Github or in community chats.
SparkyMcUnicorn 3y ago
This and/or text-generation-webui training doc are a good place to start. https://github.com/zetavg/LLaMA-LoRA-Tuner https://github.com/zetavg/LLaMA-LoRA-Tuner
ttt3ts 3y ago
https://huggingface.co/blog/stackllama https://huggingface.co/blog/stackllama You can easily finetune 7B or 15B LORA model with that on consumer GPUs.
- minimaxir 3y ago
  That blog post demonstrates that it's not "easily" finetuneable, just possible to finetune. There's many technical considerations even beyond hardware (dataset formatting, training hyperparameter nuances) that do not make it accessible to the newbie experimenting with LLMs. It's a rabbithole, and unfortunately there's no good shortcuts.
  - Solvency 3y ago
    Why have there been thousands of overnight AI/GPT startups and products in the last few months and NOT a single simple intuitive "fine tuning wizard" app? That seems like such an obvious glaring gap.
    - minimaxir 3y ago
      Because the ChatGPT API (and analogous competitors) is cheap enough that it's both faster and more cost effective to just use it instead instead of using your own model, with maybe some shenanigans to handle its shortcomings without increasing cost much if at all.. And that was before gpt-3.5-turbo-0613, which dropped the price more and is about 2-3x faster. There are startups that do finetuning on your own data, but with zero hints on how to preprocess your data and absurd costs (both upfront training and GPUs for serving inference) that's it's extremely difficult to argue from a customer business perspective compared to just using an API.
      - hyperliner 3y ago
        [dead]
    - shapefrog 3y ago
      > Why have there been thousands of overnight AI/GPT startups and products in the last few months and NOT a single simple intuitive "fine tuning wizard" app? Vapourware GPT startup inc is valued at $2bn the afternoon after you form the company and buy your first macbook. Actual usage of Ai, fine tuning etc. I can offer you $100,000 for 30% of your company if you can demonstrate a fully working product.
  - ttt3ts 3y ago
    It isn't script kiddie level but it isn't hard I finetuned a 15B parameter reddit bot with an afternoon of time and a day of training on a 3090. Bot got a few thousand Karma in a couple of days before I turned it off (proof of concept done). If all you have is an M1 or whatever, ya, you need a real workstation and depending on your use ChatGPT might be cheaper/better.
lhl 3y ago
Here's a good writeup that goes into more depth than most of the READMEs on fine-tuning: https://erichartford.com/uncensored-models https://erichartford.com/uncensored-models

Honestly, I dunno. I think most people are using lit-llama or EasyLM (on TPUs) for finetuning? QLORA is the gold standard for more affordable training. As for datasets, just look at the open datasets the best-in-class models are using, like Vicuna or https://huggingface.co/NousResearch/Nous-Hermes-13b https://huggingface.co/NousResearch/Nous-Hermes-13b Some model datasets like Manticore, Chronos or the infamous Pygmalion are more "secretive," but you can find the dataset gathering scripts on Github or in community chats.

SparkyMcUnicorn 3y ago

This and/or text-generation-webui training doc are a good place to start. https://github.com/zetavg/LLaMA-LoRA-Tuner https://github.com/zetavg/LLaMA-LoRA-Tuner

ttt3ts 3y ago

https://huggingface.co/blog/stackllama https://huggingface.co/blog/stackllama You can easily finetune 7B or 15B LORA model with that on consumer GPUs.

minimaxir 3y ago
That blog post demonstrates that it's not "easily" finetuneable, just possible to finetune. There's many technical considerations even beyond hardware (dataset formatting, training hyperparameter nuances) that do not make it accessible to the newbie experimenting with LLMs. It's a rabbithole, and unfortunately there's no good shortcuts.
- Solvency 3y ago
  Why have there been thousands of overnight AI/GPT startups and products in the last few months and NOT a single simple intuitive "fine tuning wizard" app? That seems like such an obvious glaring gap.
  - minimaxir 3y ago
    Because the ChatGPT API (and analogous competitors) is cheap enough that it's both faster and more cost effective to just use it instead instead of using your own model, with maybe some shenanigans to handle its shortcomings without increasing cost much if at all.. And that was before gpt-3.5-turbo-0613, which dropped the price more and is about 2-3x faster. There are startups that do finetuning on your own data, but with zero hints on how to preprocess your data and absurd costs (both upfront training and GPUs for serving inference) that's it's extremely difficult to argue from a customer business perspective compared to just using an API.
    - hyperliner 3y ago
      [dead]
  - shapefrog 3y ago
    > Why have there been thousands of overnight AI/GPT startups and products in the last few months and NOT a single simple intuitive "fine tuning wizard" app? Vapourware GPT startup inc is valued at $2bn the afternoon after you form the company and buy your first macbook. Actual usage of Ai, fine tuning etc. I can offer you $100,000 for 30% of your company if you can demonstrate a fully working product.
- ttt3ts 3y ago
  It isn't script kiddie level but it isn't hard I finetuned a 15B parameter reddit bot with an afternoon of time and a day of training on a 3090. Bot got a few thousand Karma in a couple of days before I turned it off (proof of concept done). If all you have is an M1 or whatever, ya, you need a real workstation and depending on your use ChatGPT might be cheaper/better.

lhl 3y ago

Here's a good writeup that goes into more depth than most of the READMEs on fine-tuning: https://erichartford.com/uncensored-models https://erichartford.com/uncensored-models