5 ms·

I’m looking forward to trying this out!

MuffinFlavored 3y ago
If this is 13B and we can assume GPT-4 is 100B+, is this even in the same ballpark as useful? Being fast at generating "something" (spitting out tokens) is one thing but... if those tokens aren't worth much (aka if the quality of the "answers" are weak), what is it really worth?
- fbhabbed 3y ago
  GPT3.5 was 175B parameters and GPT4 is larger (nobody knows how larger) and it blows GPT3.5 out of the water. So, GPT4 is, well, way more than 100B+. And yes, I share your opinion
  - jml78 3y ago
    I used chatgpt4 daily. I tried using some of these 13b models and was completely underwhelmed when compared gpt4.
    - tmalsburg2 3y ago
      I think some of these 13B models may only be pretrained and may not have benefited from RLHF. So it’s not clear how much of the difference is due to parameter count.
  - emadm 3y ago
    The ability to run it on your own private data and train/distill it from the larger models is very useful (see Orca for example: https://arxiv.org/abs/2306.02707 https://arxiv.org/abs/2306.02707). Most of the valuable data in the world is private data unsuitable to send to GPT4 and other proprietary models.
    - MuffinFlavored 3y ago
      > is very useful How? Is there not extreme correlation between "13B parameters (low/not a lot) most likely means lots of hallucinations"
      - emadm 3y ago
        What’s better a 130bn model or 10 13bn models working together.
        jmiskovic 3y ago
        The GPT4 cognitive leaps and breadth of knowledge seem impossible to replicate with any number of 13B models. 13B x10 might be the only feasible business model for your company, but that alone doesn't make it a superior option. Can you share more details on how the models work together?
  - brianjking 3y ago
    There is no information about the size of GPT-4; it could very well be smaller than GPT-3.5. Sam has said multiple times "the age of large models is over" https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/ https://www.wired.com/story/openai-ceo-sam-altman-the-age-of... Either way, the LLaMa models are quite powerful and now that we have an open, commercially usable version in 3, 7, and 13b sizes is huge. The 65b model I'm assuming is coming soon as well.
- yumraj 3y ago
  > what is it really worth? For some, nothing. For some, a lot. If you’re looking to create a ChatGPT replacement, this ain’t it. If you’re trying to learn about LLMs, how they work, capabilities vs deficiencies, private data use, etc., these are invaluable to get your hands dirty in a private environment.
  - MuffinFlavored 3y ago
    I don't get what running/researching an LLM that is really bad at hallucinating is worth.
    - yumraj 3y ago
      Don’t think production code, think MVP that may allow you to prove certain hypothesis and/or demonstrate need. That may/will allow you to raise funds which will allow you to actually pay for OpenAI’s API or maybe train your own LLM or whatever. Again, if you think in terms of MVP, these become extremely valuable.
- jwitthuhn 3y ago
  For me the big difference is that I can run llama models locally which is a different use-case entirely compared to GPT-4. Even if the weights were available, I just don't have the hardware to run something with the parameter counts of GPT-3.5+
- anon373839 3y ago
  There is now published research demonstrating that small models (LLaMA 7B) surpass GPT-4’s performance when fine-tuned on specific tasks.
  - MuffinFlavored 3y ago
    > There is now published research demonstrating that small models (LLaMA 7B) surpass GPT-4’s performance when fine-tuned on specific tasks. Got any examples? I don't understand how it can "hallucinate like crazy" for some but not for all
    - anon373839 3y ago
      1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks https://huggingface.co/papers/2305.14201 https://huggingface.co/papers/2305.14201 2. Gorilla: Large Language Model Connected with Massive APIs https://arxiv.org/abs/2305.15334 https://arxiv.org/abs/2305.15334
cypress66 3y ago
They really aren't. 30B models are much smarter, and work fine with 4 bit quantization on a 24GB GPU. On a headless system you can get the full 2048 context size. On a desktop around 1500 I think.

If this is 13B and we can assume GPT-4 is 100B+, is this even in the same ballpark as useful? Being fast at generating "something" (spitting out tokens) is one thing but... if those tokens aren't worth much (aka if the quality of the "answers" are weak), what is it really worth?

fbhabbed 3y ago
GPT3.5 was 175B parameters and GPT4 is larger (nobody knows how larger) and it blows GPT3.5 out of the water. So, GPT4 is, well, way more than 100B+. And yes, I share your opinion
- jml78 3y ago
  I used chatgpt4 daily. I tried using some of these 13b models and was completely underwhelmed when compared gpt4.
  - tmalsburg2 3y ago
    I think some of these 13B models may only be pretrained and may not have benefited from RLHF. So it’s not clear how much of the difference is due to parameter count.
- emadm 3y ago
  The ability to run it on your own private data and train/distill it from the larger models is very useful (see Orca for example: https://arxiv.org/abs/2306.02707 https://arxiv.org/abs/2306.02707). Most of the valuable data in the world is private data unsuitable to send to GPT4 and other proprietary models.
  - MuffinFlavored 3y ago
    > is very useful How? Is there not extreme correlation between "13B parameters (low/not a lot) most likely means lots of hallucinations"
    - emadm 3y ago
      What’s better a 130bn model or 10 13bn models working together.
      - jmiskovic 3y ago
        The GPT4 cognitive leaps and breadth of knowledge seem impossible to replicate with any number of 13B models. 13B x10 might be the only feasible business model for your company, but that alone doesn't make it a superior option. Can you share more details on how the models work together?
- brianjking 3y ago
  There is no information about the size of GPT-4; it could very well be smaller than GPT-3.5. Sam has said multiple times "the age of large models is over" https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/ https://www.wired.com/story/openai-ceo-sam-altman-the-age-of... Either way, the LLaMa models are quite powerful and now that we have an open, commercially usable version in 3, 7, and 13b sizes is huge. The 65b model I'm assuming is coming soon as well.
yumraj 3y ago
> what is it really worth? For some, nothing. For some, a lot. If you’re looking to create a ChatGPT replacement, this ain’t it. If you’re trying to learn about LLMs, how they work, capabilities vs deficiencies, private data use, etc., these are invaluable to get your hands dirty in a private environment.
- MuffinFlavored 3y ago
  I don't get what running/researching an LLM that is really bad at hallucinating is worth.
  - yumraj 3y ago
    Don’t think production code, think MVP that may allow you to prove certain hypothesis and/or demonstrate need. That may/will allow you to raise funds which will allow you to actually pay for OpenAI’s API or maybe train your own LLM or whatever. Again, if you think in terms of MVP, these become extremely valuable.
jwitthuhn 3y ago
For me the big difference is that I can run llama models locally which is a different use-case entirely compared to GPT-4. Even if the weights were available, I just don't have the hardware to run something with the parameter counts of GPT-3.5+
anon373839 3y ago
There is now published research demonstrating that small models (LLaMA 7B) surpass GPT-4’s performance when fine-tuned on specific tasks.
- MuffinFlavored 3y ago
  > There is now published research demonstrating that small models (LLaMA 7B) surpass GPT-4’s performance when fine-tuned on specific tasks. Got any examples? I don't understand how it can "hallucinate like crazy" for some but not for all
  - anon373839 3y ago
    1. Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks https://huggingface.co/papers/2305.14201 https://huggingface.co/papers/2305.14201 2. Gorilla: Large Language Model Connected with Massive APIs https://arxiv.org/abs/2305.15334 https://arxiv.org/abs/2305.15334

cypress66 3y ago

They really aren't. 30B models are much smarter, and work fine with 4 bit quantization on a 24GB GPU. On a headless system you can get the full 2048 context size. On a desktop around 1500 I think.