3 ms·

The value of GPT-4 also lies in its stored knowledge. A 70B model can't store that much.

lolinder 3y ago
The advantage of LLaMA 2 is that a company can fine tune it on the knowledge that they actually care about and then run it on their own hardware without paying API fees or relying on an unstable dependency that's constantly being tweaked.
- meepmorp 3y ago
  > without paying API fees or relying on an unstable dependency that's constantly being tweaked. and without handing a whole bunch of data to a 3rd party and hope they're securing it properly
  - lhl 3y ago
    Note, that just because you're hosting itself won't mean you're securing it properly... Here's a just published injection attack that only works on open source models (public model weights): https://twitter.com/random_walker/status/1683833600196714497 https://twitter.com/random_walker/status/1683833600196714497
    - huggingmouth 3y ago
      There's a world of difference between visiting links hallucinated by an unreliable ai and having a third party store (and possibly sell) every single thing you say to an ai and tying it to your identity, forever. Also there is nothing about that attack that makes it iinherently only applicable to self hosted models.
      - lhl 3y ago
        That specific attack requires adversarial inputs crafted crafted against gradients so only works against open models (requires known model weights). There are dangers that include leaking PII from your current context, but also worse if you are using the model with RAG or with other types of system access so I don't think it's as innocuous as you are assuming.
- mensetmanusman 3y ago
  That will be possible with cloud AI in the future. On prem will always be less compute capable unless you have your own GPU cluster to rival the FAANGs, that is why meta is releasing this for free.
  - lolinder 3y ago
    This is the key part of what I said: > without paying API fees or relying on an unstable dependency that's constantly being tweaked I see no evidence that this part will be possible with OpenAI. Usage fees will always be a thing because that's how they make money, and based on what I've heard from people who have actually tried to build on their APIs, I would not trust them to keep the model stable. There are always new safety features they need to add, and those changes break things.

The advantage of LLaMA 2 is that a company can fine tune it on the knowledge that they actually care about and then run it on their own hardware without paying API fees or relying on an unstable dependency that's constantly being tweaked.

meepmorp 3y ago
> without paying API fees or relying on an unstable dependency that's constantly being tweaked. and without handing a whole bunch of data to a 3rd party and hope they're securing it properly
- lhl 3y ago
  Note, that just because you're hosting itself won't mean you're securing it properly... Here's a just published injection attack that only works on open source models (public model weights): https://twitter.com/random_walker/status/1683833600196714497 https://twitter.com/random_walker/status/1683833600196714497
  - huggingmouth 3y ago
    There's a world of difference between visiting links hallucinated by an unreliable ai and having a third party store (and possibly sell) every single thing you say to an ai and tying it to your identity, forever. Also there is nothing about that attack that makes it iinherently only applicable to self hosted models.
    - lhl 3y ago
      That specific attack requires adversarial inputs crafted crafted against gradients so only works against open models (requires known model weights). There are dangers that include leaking PII from your current context, but also worse if you are using the model with RAG or with other types of system access so I don't think it's as innocuous as you are assuming.
mensetmanusman 3y ago
That will be possible with cloud AI in the future. On prem will always be less compute capable unless you have your own GPU cluster to rival the FAANGs, that is why meta is releasing this for free.
- lolinder 3y ago
  This is the key part of what I said: > without paying API fees or relying on an unstable dependency that's constantly being tweaked I see no evidence that this part will be possible with OpenAI. Usage fees will always be a thing because that's how they make money, and based on what I've heard from people who have actually tried to build on their APIs, I would not trust them to keep the model stable. There are always new safety features they need to add, and those changes break things.