6 ms·
Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions
by colesantiago 3y ago
Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4.
Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI.
Not to mention their stock is up and they are worth $1TN, again.
Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Analytica, and last of all caused a genocide in Myanmar.
Goes to show that nobody cares about all of these scandals and just moves on onto the future, allowing Facebook to still collect all this data for their models.
If any other startup or mid sized company had at least two of these large scandals, they would be dead in the water.
- austinpena 3y agoI'm really curious what their goal is
- asylteltine 3y ago[dead]
- Ruq 3y agoZuck just wants another robot to talk to.
- throwup238 3y agoPoor guy just wants a friend that won't sell him out to the FTC or some movie producers.
- hendersoon 3y agoTheir goals are clear, dominance and stockholder value. What I'm curious about is how they plan to monetize it.
- refulgentis 3y agoUse it in products, ex. the chatbots
- regimeld 3y agoCommoditize your compliments.
- refulgentis 3y agoSame as MS, in the game, in the conversation, and ensuring next-gen search margins approximate 0.
- Arainach 3y agoDisclaimer: I do not work at Meta, but I work at a large tech company which competes with them. I don't work in AI, although if my VP asks don't tell them I said that or they might lay me off. Multiple of their major competitors/other large tech companies are trying to monetize LLMs. OpenAI maneuvering an early lead into a dominant position would be another potential major competitor. If releasing these models slows or hurts them that is in and itself a benefit.
- michaelt 3y agoWhy? What benefit is there to grabbing market share from your competitors... in a business you don't even want to be in? By that logic you could justify any bizarre business decision. Should Google launch a social network, to hurt their competitor Facebook? Should Facebook, Amazon and Microsoft each launch a phone?
- Filligree 3y ago> Should Google launch a social network, to hurt their competitor Facebook? I mean, Google did launch a social network, to hurt their competitor Facebook. It was a whole thing. It was even a really nice system, eventually.
- wanderingstan 3y agoAnd it turned out that Facebook had quite a moat with network effects. OpenAI doesn’t have such a moat, which may be what Meta is wanting to expose.
- esafak 3y agoGoogle botched the launch, and they never nurture products after launch anyway. Google+ could have been more successful.
- zaat 3y agoI enjoyed using Google plus more than any other social network, and managed to create new connections and/or have standard, authentic, real conversations with people I didn't know, most of them ordinary people with shared interests that I would probably wouldn't meet otherwise, some of them are people I can't believe I could connect with directly in any other way - newspapers and news sites editors, major SDK developers. And even with Kevin Kelly.
- tsunamifury 3y agoTheir goal is to counter the competition. You rarely should pick the exact same strategy as your competitor and count on out gunning them, rather you should counter them. OpenAI is ironically closed, well meta will be open then. If you can't beat them, you should try to degrade down the competitors value case. Its a smart move IMO
- dsabanin 3y agoI think Meta's goal is to subvert Google, MS and OpenAI, after realizing it's not positioned well to compete with them commercially.
- api 3y agoCould also be that these smaller models are a loss leader or advertisement for a future product or service... like a big brother to Llama3 that's commercial.
- idkyall 3y agoI believe there were rumors they are developing a commercial model: e.g. https://www.ft.com/content/01fd640e-0c6b-4542-b82b-20afb203f271 https://www.ft.com/content/01fd640e-0c6b-4542-b82b-20afb203f...
- akozak 3y agoI would guess mindshare in a crowded field, ie discussion threads just like this one that help with recruiting and tech reputation after a bummer ~8 years. (It's best not to overestimate the complexity/# of layers in a bigco's strategic thinking.)
- jedberg 3y agoCommoditizing your complement. If all your competitors need a key technology to get ahead, you make a cheap/free version of it so that they can't use it as a competitive advantage.
- nindalf 3y agoThe complement being the metaverse. You can’t handcraft the metaverse because it would be infeasible. If LLMs are a commodity that everyone has access to, then it can be done on the cheap. Put another way - if OpenAI were the only game in town how much would they be charging for their product? They’re competing on price because competitors exist. Now imagine the price if a hypothetical high quality open source model existed that can customers can use for “free”. That’s the future Meta wants. They weren’t getting rich selling shovels like cloud providers are, they want everyone digging. And everyone digs when the shovels are free.
- aantix 3y agoTo undermine the momentum of OpenAI. If Meta were at the forefront, these models would not be openly available. They are scrambling.
- blackoil 3y agoDevil's advocate: they have to build it anyway for Meta verse and in general. Management has no interest in going into cloud business. They had Parse long time back but that is done. So why not to release it. They are getting goodwill/mindshare, may set up industry standard and get community benefit. It isn't very different from React, Torch etc.
- ttul 3y agoIf you want to employ the top ML researchers, you have to give them what they want, which is often the ability to share their discoveries with the world. Making Llama-N open may not be Zuckerberg‘s preference. It’s possible the researchers demanded it.
- ppsreejith 3y agoIf you go by what Zuck says, he calls this out in previous earnings reports and interviews[1]. It mainly boils down to 2 things: 1. Similar to other initiatives (mainly opencompute but also PyTorch, React etc), community improvements help them improve their own infra and helps attract talent. 2. Helping people create better content ultimately improves quality of content on their platforms (Both FoA & RL) Sources: [1]Interview with verge: https://www.theverge.com/23889057/mark-zuckerberg-meta-ai-elon-musk-threads-quest-interview-decoder https://www.theverge.com/23889057/mark-zuckerberg-meta-ai-el... . Search for "regulatory capture right now with AI" > Zuck: ... And we believe that it’s generally positive to open-source a lot of our infrastructure for a few reasons. One is that we don’t have a cloud business, right? So it’s not like we’re selling access to the infrastructure, so giving it away is fine. And then, when we do give it away, we generally benefit from innovation from the ecosystem, and when other people adopt the stuff, it increases volume and drives down prices. > Interviewer: Like PyTorch, for example? > Zuck: When I was talking about driving down prices, I was thinking about stuff like Open Compute, where we open-sourced our server designs, and now the factories that are making those kinds of servers can generate way more of them because other companies like Amazon and others are ordering the same designs, that drives down the price for everyone, which is good.
- PheonixPharts 3y agoI imagine their goal is to simultaneously show that Meta is still SotA when it comes to AI and at the same time feed a community of people who will work for free to essentially undermine OpenAI's competitive advantage and make life worse for Google since at the very least LLMs tend to be a better search engine for most topics. There's far more risk if Meta were to try to directly compete with OpenAI and Microsoft on this. They'd have to manage the infra, work to acquire customers, etc, etc on top of building these massive models. If it's not a space they really want to be in, it's a space they can easily disrupt. Meta's late game realization was that Google owned the web via search and Apple took over a lot of the mobile space with their walled garden. I suspect Meta's view now is that it's much easier to just prevent something like this from happening with AI early on.
- wrsh07 3y agoRule 5: commodify your complement Content generation is complementary to most of meta's apps and projects
- elorant 3y agoPrevent OpenAI from dominating the market, and at the same time have the research community enhance your models and identify key use cases.
- pennomi 3y agoMeta basically got a ton of free R&D that directly applies to their model architecture. Their next generation AIs will always benefit from the techniques/processes developed by the clever researchers and hobbyists out there.
- megaman821 3y agoThey were going to make most this anyway for Instagram filters, chat stickers, internal coding tools, VR world generation, content moderation, etc. Might as well do a little bit extra work to open source it since it doesn't really compete with anything Meta is selling.
- make3 3y agowith pytorch (& so many open publications), Meta has had a unimaginably strong contribution to ai for a while
- fragmede 3y agoModel available, not open source. These models aren't open source because we don't have access to the data sets, nor the full code to train them, so we can't recreate the models even if we had the GPU time available to recreate them.
- colesantiago 3y agoEveryone using AI in production is using Pytorch by Meta. Which is open source. I do not know anybody important in the AI space apart from Google using TensorFlow.
- Philpax 3y agoThat may be true, but it's largely irrelevant. The ML framework in use has no bearing on whether or not you have the data required to reproduce the model being trained with that framework.
- colesantiago 3y agoDo you and the GP have 350K GPUs and quality data to reproduce 1:1 whatever Facebook releases in their repos? Even if you want to reproduce the model and they give you the data, you would need to do this at Facebook scale, so you and the GP are just making moot points all around. https://about.fb.com/news/2023/05/metas-infrastructure-for-ai/ https://about.fb.com/news/2023/05/metas-infrastructure-for-a... https://www.theregister.com/2024/01/20/metas_ai_plans/ https://www.theregister.com/2024/01/20/metas_ai_plans/ The fact that these models are coming from Meta in the open rather than Google which releases only papers with no model tell's me that Meta's models is open enough for everyone to use. Besides, everyone using the Pytorch framework benefits Meta in the same way they were originally founded as a company: Network effects It's relevant.
- Philpax 3y agoThere are organisations that are capable of reproduction (e.g. EleutherAI), but yes, you're right, not having the data is largely irrelevant for most users. The thing that bothers me more is that it's not actually an open-source licence; there are restrictions on what you can do with it, and whatever you do with the model is subject to those restrictions. It's still very useful and I'm not opposed to them releasing it under that licence (they need to recoup the costs somehow), but "open-source" (or even "open") it is not.
- WhackyIdeas 3y agoIt does seem like the nicest thing Facebook have ever done by giving so much to the open source LLM scene, I know that it might have been started by a leaker, but they have given so much voluntarily. I mean don’t get me wrong, I don’t like the company but I do really like some of the choices they have made recently. But I do wonder in the back of my mind why. And I should be suspicious of their angle and I will keep thinking about it. Is it paranoid to think that maybe their angle is putting almost some kind of metadata by style of code being unique to different machines that they can trace generated code to different people? Is that their angle or am I biased in remembering who they have been for the past decade?
- cm2012 3y agoCambridge Analytica is not a real scandal (did not affect any elections), and FB did not cause a genocide in Myanmar (they were a popular message board during a genocide, which is not the same thing)