4 ms·

OpenAI is going to be left in the dust by (actual) open models. Llama 2 is already reaching GPT-3 levels, and can run inference on consumer hardware. Crazy how

OpenAI is going to be left in the dust by (actual) open models. Llama 2 is already reaching GPT-3 levels, and can run inference on consumer hardware. Crazy how fast that flipped.

nomel 3y ago
What enables this? There's a huge gap between GPT-3.5 and 4, put there by a massive amount of money, from my understanding. To compete, with open source projects being less well funded, I would assume that orders of magnitude improvements in training cost would be required. What do you see driving this, and who do you see paying for it? If Meta, or anyone else, gets something that beats GPT-4, I would naively assume they would monetize it. If someopen source effort manages to beat GPT-4, then I assume some well funded, profit seeking, entity will dump orders of magnitude more money into whatever enabled the open source offerings to compete. I don't think GPT-4 is the pinnacle of OpenAI, or that their funding will run dry, so they shouldn't be viewed as a static entity.
- ramesh31 3y ago
  >Or, are you suggesting that GPT-4 is the pinnacle of OpenAI, or that their funding will run dry? My bet is that Meta has pivoted almost entirely to this space with their R&D in the last six months. Llama 2 is spectacular. And with its' success, there will undoubtedly be more. They also happen to have access to limitless amounts of compute, cash, and engineering that puts OpenAI to shame. This could finally be their chance to create a platform for real. And the open source community and startups will benefit off of that.
  - deleted 3y ago
    [deleted]
  - nomel 3y ago
    What motive do you see for them releasing these models for free, especially with the massive increase in cost associated with catching up?
    - ramesh31 3y ago
      > What motive do you see for them releasing these models for free, especially with the massive increase in cost associated with catching up? Owning a platform. Zuck's dream. They put tons of money into these models, nurture an ecosystem of companies built around them, and then start gradually figuring out a licensing model for the ones that take off.
    - littlestymaar 3y ago
      Kill Op*nAI, for starter. If they see it as a threat, commoditizing the tech is a great way to get rid of them at a reasonable cost.
      - deleted 3y ago
        [deleted]
  - nmfisher 3y ago
    I haven't tried Llama 2 yet - what specifically makes you say it's spectacular? The licensing or the model itself?
flangola7 3y ago
Llama really isn't open source, at least not in the sense of FOSS licenses like GPL or MIT. It comes with a number of use-case conditions and gives Meta many avenues to revoke a license if they feel like it. They also have a hard cap on the number of allowed users you may have using your Llama-based product above which you must seek further Meta approval. Furthermore, Llama remains well below GPT-3 on human rated tests such as programming, and GPT-3 is already over three years old. It is also misleading to suggest Llama 2 can be ran on consumer hardware - the smaller and quantized models can but those are even more lacking in capability. Full power Llama 2 still requires multiple kilowatts of electricity and $10,000+ of compute hardware per inference session. OpenAI does not have a moat, but they do have a very high wall.
- Sosh101 3y ago
  > and $10,000+ of compute hardware per inference session. What hardware would you need to run it at home?
  - ramesh31 3y ago
    >What hardware would you need to run it at home? Step 1: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/main/llama-2-7b-chat.ggmlv3.q4_0.bin https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/ma... Step 2: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp Step 3: you're welcome
    - Sosh101 3y ago
      That's very helpful, thank you.
  - avion23 3y ago
    > and $10,000+ of compute hardware per inference session. That is not true. A common macbook with lots of RAM (>32GB) is enough. Or any x86 computer with lots of RAM. llama.cpp is CPU only and quite fast
- ramesh31 3y ago

OpenAI is going to be left in the dust by (actual) open models. Llama 2 is already reaching GPT-3 levels, and can run inference on consumer hardware. Crazy how fast that flipped.

nomel 3y ago

What enables this? There's a huge gap between GPT-3.5 and 4, put there by a massive amount of money, from my understanding. To compete, with open source projects being less well funded, I would assume that orders of magnitude improvements in training cost would be required. What do you see driving this, and who do you see paying for it? If Meta, or anyone else, gets something that beats GPT-4, I would naively assume they would monetize it. If someopen source effort manages to beat GPT-4, then I assume some well funded, profit seeking, entity will dump orders of magnitude more money into whatever enabled the open source offerings to compete. I don't think GPT-4 is the pinnacle of OpenAI, or that their funding will run dry, so they shouldn't be viewed as a static entity.

ramesh31 3y ago
>Or, are you suggesting that GPT-4 is the pinnacle of OpenAI, or that their funding will run dry? My bet is that Meta has pivoted almost entirely to this space with their R&D in the last six months. Llama 2 is spectacular. And with its' success, there will undoubtedly be more. They also happen to have access to limitless amounts of compute, cash, and engineering that puts OpenAI to shame. This could finally be their chance to create a platform for real. And the open source community and startups will benefit off of that.
- deleted 3y ago
  [deleted]
- nomel 3y ago
  What motive do you see for them releasing these models for free, especially with the massive increase in cost associated with catching up?
  - ramesh31 3y ago
    > What motive do you see for them releasing these models for free, especially with the massive increase in cost associated with catching up? Owning a platform. Zuck's dream. They put tons of money into these models, nurture an ecosystem of companies built around them, and then start gradually figuring out a licensing model for the ones that take off.
  - littlestymaar 3y ago
    Kill Op*nAI, for starter. If they see it as a threat, commoditizing the tech is a great way to get rid of them at a reasonable cost.
    - deleted 3y ago
      [deleted]
- nmfisher 3y ago
  I haven't tried Llama 2 yet - what specifically makes you say it's spectacular? The licensing or the model itself?

flangola7 3y ago

Llama really isn't open source, at least not in the sense of FOSS licenses like GPL or MIT. It comes with a number of use-case conditions and gives Meta many avenues to revoke a license if they feel like it. They also have a hard cap on the number of allowed users you may have using your Llama-based product above which you must seek further Meta approval. Furthermore, Llama remains well below GPT-3 on human rated tests such as programming, and GPT-3 is already over three years old. It is also misleading to suggest Llama 2 can be ran on consumer hardware - the smaller and quantized models can but those are even more lacking in capability. Full power Llama 2 still requires multiple kilowatts of electricity and $10,000+ of compute hardware per inference session. OpenAI does not have a moat, but they do have a very high wall.

Sosh101 3y ago
> and $10,000+ of compute hardware per inference session. What hardware would you need to run it at home?
- ramesh31 3y ago
  >What hardware would you need to run it at home? Step 1: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/main/llama-2-7b-chat.ggmlv3.q4_0.bin https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/blob/ma... Step 2: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp Step 3: you're welcome
  - Sosh101 3y ago
    That's very helpful, thank you.
- avion23 3y ago
  > and $10,000+ of compute hardware per inference session. That is not true. A common macbook with lots of RAM (>32GB) is enough. Or any x86 computer with lots of RAM. llama.cpp is CPU only and quite fast
ramesh31 3y ago