4 ms·

Stating independence as the advantage of Llama 3.1 is a bit funny. Without the huge amount of computational resources from Meta, Llama 3.1 won't be possible. We

Stating independence as the advantage of Llama 3.1 is a bit funny. Without the huge amount of computational resources from Meta, Llama 3.1 won't be possible. We are still dependent on certain big companys' "good" willness to be able to enjoy the benefits of open source.

jsheard 2y ago
May as well enjoy it while it lasts, but yeah it echoes the precarious state that VR is currently in, where Meta is more or less propping up the entire consumer market with their willingness to burn billions of dollars on Reality Lab every quarter, and anyone investing in VR development just has to hope that rug doesn't get pulled from under them any time soon. Llama 3.1 isn't going away of course, but how much are you willing on gamble on Llama 3.1 possibly being the last SOTA model to be released this freely?
- sReinwald 2y ago
  > [...] the precarious state that VR is currently in, where Meta is more or less propping up the entire consumer market with their willingness to burn billions of dollars on Reality Lab every quarter, They don't quite seem so happy to do that for much longer. https://www.businessinsider.com/mark-zuckerberg-cuts-spending-reality-labs-meta-report-2024-7 https://www.businessinsider.com/mark-zuckerberg-cuts-spendin...
  - jsheard 2y ago
    They could buy a lot of H100s for the roughly $4 billion they dump into VR/Metaverse each quarter...
    - phyrex 2y ago
      They are complimentary technologies. Think holodeck.
idle_zealot 2y ago
For the release, yes. But from now onwards everyone will benefit from Llama 3.1 regardless of Meta's involvement. That's the advantage; I can build something with Llama 3.1 and it will keep working indefinitely without any dependence on another party.
tkellogg 2y ago
Yeah, that's a good point. On the other hand, the model is out there, irrevocably so, and you can use knowledge distillation techniques to train other weaker models. So yeah, we're not fully independent, but the situation is significantly improved.
yieldcrv 2y ago
eh synthetic data sets based on llama 3.1 will train some fun models
t0md4n 2y ago
I’ve always wondered if it’s possible to pool together small amounts of resources from users e.g. similar to torrenting with networking, but with compute and use it to host a large model like this one. If the community wants open source to be the way, I believe this would be the only way to get these large models accessible to users while the community stay in semi-control of the platform. Edit: would also work with training I'm assuming?

jsheard 2y ago

May as well enjoy it while it lasts, but yeah it echoes the precarious state that VR is currently in, where Meta is more or less propping up the entire consumer market with their willingness to burn billions of dollars on Reality Lab every quarter, and anyone investing in VR development just has to hope that rug doesn't get pulled from under them any time soon. Llama 3.1 isn't going away of course, but how much are you willing on gamble on Llama 3.1 possibly being the last SOTA model to be released this freely?

sReinwald 2y ago
> [...] the precarious state that VR is currently in, where Meta is more or less propping up the entire consumer market with their willingness to burn billions of dollars on Reality Lab every quarter, They don't quite seem so happy to do that for much longer. https://www.businessinsider.com/mark-zuckerberg-cuts-spending-reality-labs-meta-report-2024-7 https://www.businessinsider.com/mark-zuckerberg-cuts-spendin...
- jsheard 2y ago
  They could buy a lot of H100s for the roughly $4 billion they dump into VR/Metaverse each quarter...
  - phyrex 2y ago
    They are complimentary technologies. Think holodeck.

idle_zealot 2y ago

For the release, yes. But from now onwards everyone will benefit from Llama 3.1 regardless of Meta's involvement. That's the advantage; I can build something with Llama 3.1 and it will keep working indefinitely without any dependence on another party.

tkellogg 2y ago

Yeah, that's a good point. On the other hand, the model is out there, irrevocably so, and you can use knowledge distillation techniques to train other weaker models. So yeah, we're not fully independent, but the situation is significantly improved.

yieldcrv 2y ago

eh synthetic data sets based on llama 3.1 will train some fun models

t0md4n 2y ago

I’ve always wondered if it’s possible to pool together small amounts of resources from users e.g. similar to torrenting with networking, but with compute and use it to host a large model like this one. If the community wants open source to be the way, I believe this would be the only way to get these large models accessible to users while the community stay in semi-control of the platform. Edit: would also work with training I'm assuming?