4 ms·
This is obviously good news, but __personally__ I feel the open-source models are just trying to catch up with whoever the market leader is, based on some bench
by Oras 2y ago
This is obviously good news, but __personally__ I feel the open-source models are just trying to catch up with whoever the market leader is, based on some benchmarks.
The actual problem is running these models. Very few companies can afford the hardware to run these models privately. If you run them in the cloud, then I don't see any potential financial gain for any company to fine-tune these huge models just to catch up with OpenAI or Anthropic, when you can probably get a much better deal by fine-tuning the closed-source models.
Also this point:
> We need to protect our data. Many organizations handle sensitive data that they need to secure and can’t send to closed models over cloud APIs.
First, it's ironic that Meta is talking about privacy. Second, most companies will run these models in the cloud anyway. You can run OpenAI via Azure Enterprise and Anthropic on AWS Bedrock.
- deleted 2y ago[deleted]
- simonw 2y ago"Very few companies can afford the hardware to run these models privately." I can run Llama 3 70B on my (64GB RAM M2) laptop. I haven't tried 3.1 yet but I expect to be able to run that 70B model too. As for the 405B model, the Llama 3.1 announcement says: > To support large-scale production inference for a model at the scale of the 405B, we quantized our models from 16-bit (BF16) to 8-bit (FP8) numerics, effectively lowering the compute requirements needed and allowing the model to run within a single server node.