7 ms·
> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. Bravo! While I don't
by cs702 2y ago
> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models.
Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. With this release, they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO for pretty much everyone -- well, everyone except Meta's key competitors.[a] THANK YOU ZUCK.
Meanwhile, the business-minded people at Meta surely won't mind if the release of these frontier models to the public happens to completely mess up the AI plans of competitors like OpenAI/Microsoft, Google, Anthropic, etc. Come to think of it, the negative impact on such competitors was likely a key motivation for releasing the new models.
---
[a] The license is not open to the handful of companies worldwide which have more than 700M users.
- swyx 2y ago> the AI folks at Meta deserve our praise and gratitude We interviewed Thomas who led Llama 2 and 3 post training here in case you want to hear from someone closer to the ground on the models https://www.latent.space/p/llama-3 https://www.latent.space/p/llama-3
- throwaway_2494 2y ago> We’re releasing Llama 3.1 405B Is it possible to run this with ollama?
- jessechin 2y agoSure, if you have a H100 cluster. If you quant it to int4 you might get away with using only 4 H100 GPUs!
- sheepscreek 2y agoAssuming $25k a pop, that’s at least $100k in just the GPUs alone. Throw in their linking technology (NVLink) and cost for the remaining parts, won’t be surprised if you’re looking at $150k for such a cluster. Which is not bad to be honest, for something at this scale. Can anyone share the cost of their pre-built clusters, they’ve recently started selling? (sorry feeling lazy to research atm, I might do that later when I have more time).
- vorticalbox 2y agoIf you have the ram for it. Ollama will offload as many layers as it can to the gpu then the rest will run on the cpu/ram.
- deleted 2y ago[deleted]
- Havoc 2y agoIf you want your first token around tomorrow lunch sure
- tambourine_man 2y agoPraising is good. Gratitude is a bit much. They got this big by selling user generated content and private info to the highest bidder. Often through questionable means. Also, the underdog always touts Open Source and standards, so it’s good to remain skeptical when/if tables turn.
- sheepscreek 2y agoAll said and done, it is a very expensive and balsy way to undercut competitors. They’ve spent > $5B on hardware alone, much of which will depreciate in value quickly. Pretty sure the only reason Meta’s managed to do this is because of Zuck’s iron grip on the board (majority voting rights). This is great for Open Source and regular people though!
- wrsh07 2y agoZuck made a bet when they provisioned for reels to buy enough GPUs to be able to spin up another reels-sized service. Llama is probably just running on spare capacity (I mean, sure, they've kept increasing capex, but if they're worried about an llm-based fb competitor they sort of have to in order to enact their copycat strategy)
- fractalf 2y agoWell, he didn't do it to be "nice", you can be sure about that. Obviously they see a financial gain somewhere/sometime
- tambourine_man 2y agoAt Meta level, spending $5B to stay competitive is not balsy. It’s a bargain.
- ricardo81 2y ago>selling user generated content and private info to the highest bidder Was always their modus operandi, surely. How else would they have survived. Thanks for returning everyone else;s content and never mind all the content stealing your platform did.
- 2y ago
- germinalphrase 2y ago"Come to think of it, the negative impact on such competitors was likely a key motivation for releasing the new models." "Commoditize Your Complement" is often cited here: https://gwern.net/complement https://gwern.net/complement
- tintor 2y ago> they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO It is still far from zero.
- cs702 2y agoIf the model is already pretrained, there's no need to pretrain it, so the cost of pretraining is zero.
- moffkalast 2y agoYeah but you only have the one model, and so far it seems to be only good on paper.
- pwdisswordfishd 2y agoMakes me wonder why he's really doing this. Zuckerberg being Zuckerberg, it can't be out of any genuine sense of altruism. Probably just wants to crush all competitors before he monetizes the next generation of Meta AI.
- spiralk 2y agoIts certainly not altruism. Given that Facebook/Meta owns the largest user data collection systems, any advancement in AI ultimately strengthens their business model (which is still mostly collecting private user data, amassing large user datasets, and selling targeting ads). There is a demo video that shows a user wearing a Quest VR headset and asks the AI "what do you see" and it interprets everything around it. Then, "what goes well with these shorts"... You can see where this is going. Wearing headsets with AIs monitoring everything the users see and collecting even more data is becoming normalized. Imagine the private data harvesting capabilities of the internet but anywhere in the physical world. People need not even choose to wear a Meta headset, simply passing a user with a Meta headset in public will be enough to have private data collected. This will be the inevitable result of vision models improvements integrated into mobile VR/AR headsets.
- goatlover 2y agoThat's very dystopian. It's bad enough having cameras everywhere now. I never opted in to being recorded.
- warkdarrior 2y agoThat sounds fantastic. If they make the Meta headset easy to wear and somewhat fashionable (closer to eyeglass than to a motorcycle helmet), I'd take it everywhere and record everything. Give me a retrospective search and conferences/meetings will be so much easier (I am terrible with names).
- meroes 2y agoI wouldn’t even say hi alone my name to someone wearing a Meta headset out in public. And if facial recognition becomes that common for wearers, most of the population is going to adorn something to prevent that. And if it’s at work, I’m not working there and I have to think many would agree. Coworkers don’t and wouldn’t tolerate coworkers taking videos or pictures of them.
- troupo 2y agoThere's nothing open source about it. It's a proprietary dump of data you can't replicate or verify. What were the sources? What datasets it was trained on? What are the training parameters? And so on and so on
- advael 2y agoLook, absolutely zero people in the world should trust any tech company when they say they care about or will keep commitments to the open-source ecosystem in any capacity. Nevertheless, it is occasionally strategic for them to do so, and there can be ancillary benefits for said ecosystem in those moments where this is the best play for them to harm their competitors For now, Meta seems to release Llama models in ways that don't significantly lock people into their infrastructure. If that ever stops being the case, you should fork rather than trust their judgment. I say this knowing full well that most of the internet is on AWS or GCP, most brick and mortar businesses use Windows, and carrying a proprietary smartphone is essentially required to participate in many aspects of the modern economy. All of this is a mistake. You can't resist all lock-in. The players involved effectively run the world. You should still try where you can, and we should still be happy when tech companies either slip up or make the momentary strategic decision to make this easier
- ori_b 2y ago> If that ever stops being the case, you should fork rather than trust their judgment. Fork what? The secret sauce is in the training data and infrastructure. I don't think either of those is currently open.
- quasse 2y agoI'm just a lowly outsider to the AI space, but calling these open source models seems kind of like calling a compiled binary open source. If you don't have a way to replicate what they did to create the model, it seems more like freeware than open source.
- advael 2y agoAs an ML researcher, I agree. Meta doesn't include adequate information to replicate the models, and from the perspective of fundamental research, the interest that big tech companies have taken in this field has been a significant impediment to independent researchers, despite the fact that they are undeniably producing groundbreaking results in many respects, due to this fundamental lack of openness This should also make everyone very skeptical of any claim they are making, from benchmark results to the legalities involved in their training process to the prospect of future progress on these models. Without being able to vet their results against the same datasets they're using, there is no way to verify what they're saying, and the credulity that otherwise smart people have been exhibiting in this space has been baffling to me As a developer, if you have a working Llama model, including the source code and weights, and it's crucial for something you're building or have already built, it's still fundamentally a good thing that Meta isn't gating it behind an API and if they went away tomorrow, you could still use, self-host, retrain, and study the models
- sandworm101 2y ago>> Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. Nope. Not one bit. Supporting F/OSS when it suits you in one area and then being totally dismissive of it in every other area should not be lauded. How about open sourcing some of FB's VR efforts?
- zmmmmm 2y agoThey open sourced Gear VR. The are a stark contrast to other players in terms of how they have built everything on open standards (OpenXR, WebXR, etc), and they have just opened their platform by allowing third parties to build and customise it to make their own commercial offerings. Not open source, but a quite a contrast to every other player in that industry so far.
- y04nn 2y agoDon't be fooled, it is a "embrace extend extinguish" strategy. Once they have enough usage and be the default standard they will start to find any possible ways to make you pay.
- war321 2y agoHasn't really happened with PyTorch or any of their other open sourced releases tbh.
- GreenWatermelon 2y agoCredits where due: Facebook didn't do that with React or PyTorch. Meta will reap benefit for sure, but they don't seem to be betting on selling the model itself, rather they will benefit from being at the forefront of a new ecosystem.
- tyler-jn 2y agoSo far, it seems like this release has done ~nothing to the stock price for GOOGL/MSFT, which we all know has been propped up largely on the basis of their AI plans. So it's probably premature to say that this has messed it up for them.