3 ms·

Fine-tuning is a much better proposition than you’re giving it credit for. Papers are coming out demonstrating that 7B parameter models can outperform GPT-4’s q

Fine-tuning is a much better proposition than you’re giving it credit for. Papers are coming out demonstrating that 7B parameter models can outperform GPT-4’s quality when trained on a limited set of tasks. Yet, a 7B model offers comparatively cheap and fast inference. Furthermore, for a lot of use cases, few-shot prompting is infeasible because you need to supply 2-3k tokens worth of few-shot examples with every prompt in order to fully specify the behavior you want. (As an example, think of long-form summarization where you want the summary to adhere to certain rules.)

sgu999 3y ago
What kind of tasks? Could you give some links to the papers you're referring to?
- anon373839 3y ago
  Sure, here are two: 1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks https://huggingface.co/papers/2305.14201 https://huggingface.co/papers/2305.14201 2. Gorilla: Large Language Model Connected with Massive APIs https://arxiv.org/abs/2305.15334 https://arxiv.org/abs/2305.15334 Consider also these 2 papers supporting the feasibility of fine-tuning: 3. LIMA: Less Is More for Alignment [showing that a very small number of high quality examples is sufficient to align a base model] https://arxiv.org/abs/2305.11206 https://arxiv.org/abs/2305.11206 4. QLoRA: Efficient Finetuning of Quantized LLMs [showing that LLMs can now be fine-tuned quickly on consumer-grade GPUs] https://arxiv.org/abs/2305.14314 https://arxiv.org/abs/2305.14314 —- Adding up these developments (all of which occurred during the span of one week), I don’t see how huge, slow, general-purpose models maintain their relevance in the long term, when a lean, domain-focused model is right there within reach of every application developer.
  - fareesh 3y ago
    Sorry I should have been more specific - I was limiting my question to the bigger models. The smaller (~7B) models are feasible with these approaches.
    - anon373839 3y ago
      Ah. Perhaps the larger models will find use in in-house deployments where companies want their employees to have access to ChatGPT-like general purpose assistants, but want to prevent data from leaving their premises. LIMA shows the LLaMA 65B hitting a quality level somewhere between DaVinci-003 and GPT-4 with minimal alignment, so the base models are probably powerful enough already for this to work. Just speculating.
    - MacsHeadroom 3y ago
      One benefit of finetuning larger models, like 65B, is to free up limited context space vs few-shot prompting. If you want a specific kind of interaction with the model then you could take up 1/3rd of the 2048 token context window with few-shot or you could simply finetune it with QLoRA for a few hours on a consumer GPU and then get to use the full 2048 context with the finetuned model.
  - sgu999 3y ago
    Wow, thanks a lot! :)

sgu999 3y ago

What kind of tasks? Could you give some links to the papers you're referring to?

anon373839 3y ago
Sure, here are two: 1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks https://huggingface.co/papers/2305.14201 https://huggingface.co/papers/2305.14201 2. Gorilla: Large Language Model Connected with Massive APIs https://arxiv.org/abs/2305.15334 https://arxiv.org/abs/2305.15334 Consider also these 2 papers supporting the feasibility of fine-tuning: 3. LIMA: Less Is More for Alignment [showing that a very small number of high quality examples is sufficient to align a base model] https://arxiv.org/abs/2305.11206 https://arxiv.org/abs/2305.11206 4. QLoRA: Efficient Finetuning of Quantized LLMs [showing that LLMs can now be fine-tuned quickly on consumer-grade GPUs] https://arxiv.org/abs/2305.14314 https://arxiv.org/abs/2305.14314 —- Adding up these developments (all of which occurred during the span of one week), I don’t see how huge, slow, general-purpose models maintain their relevance in the long term, when a lean, domain-focused model is right there within reach of every application developer.
- fareesh 3y ago
  Sorry I should have been more specific - I was limiting my question to the bigger models. The smaller (~7B) models are feasible with these approaches.
  - anon373839 3y ago
    Ah. Perhaps the larger models will find use in in-house deployments where companies want their employees to have access to ChatGPT-like general purpose assistants, but want to prevent data from leaving their premises. LIMA shows the LLaMA 65B hitting a quality level somewhere between DaVinci-003 and GPT-4 with minimal alignment, so the base models are probably powerful enough already for this to work. Just speculating.
  - MacsHeadroom 3y ago
    One benefit of finetuning larger models, like 65B, is to free up limited context space vs few-shot prompting. If you want a specific kind of interaction with the model then you could take up 1/3rd of the 2048 token context window with few-shot or you could simply finetune it with QLoRA for a few hours on a consumer GPU and then get to use the full 2048 context with the finetuned model.
- sgu999 3y ago
  Wow, thanks a lot! :)