9 ms·
Meta's open AI hardware vision
- dudus 2y agoVery impressive. Not sure if worth $40 Billion. But very impressive nonetheless.
- flkenosad 2y agoThat made me laugh. $40 billion is absurd.
- airstrike 2y agoBeats buying Twitter, that's for sure.
- throwaway48476 2y agoTwitter was always very overvalued because it's real value case can't be expressed in dollar terms.
- andrewstuart 2y agoFree and open LLMs will be compelling to many of users that cannot or do not want to use online services. It’s good that a large company like Meta seems to be pushing it forward with openllama 3.2. Open, quality LLMs must lead to some tense discussions at OpenAI and Anthropic about whether they should be open or closed. OpenAI Whisper is open source and it’s extremely good although it appeared to me the online version is much faster.
- blihp 2y agoLikely not tense as it seems pretty clear where everyone stands on this. OpenAI and Anthropic are likely having conversations along the lines of 'move faster and build the moat!' while Meta is having a conversation along the lines of 'move faster and destroy the moat!' It's not that Meta has anything against moats, it's just that they've seen how it works when they try to build their moat on top of someone else's (i.e. Apple and Google re: mobile) and it didn't work out too well.
- andrewstuart 2y agoSo they’re trying to be what Android is to the iPhone?
- LorenDB 2y agoPrecisely, and that extends to their XR ambitions as well: https://www.meta.com/blog/quest/meta-horizon-os-open-hardware-ecosystem-asus-republic-gamers-lenovo-xbox/ https://www.meta.com/blog/quest/meta-horizon-os-open-hardwar...
- threeseed 2y agoAlso the EU is far more aggressive and opinionated about what they want to see in the market than during the smartphone era. And this is spreading across the world. So being committed to an open, standards-based strategy is an easy way for them to avoid most of the risks.
- jameslk 2y agoThat title is most definitely a swipe at OpenAI. It seems there’s a theme of two types of platform companies: the closed one that has the “premium” platform, and the open one that gets market share by commodifying the platform. macOS vs Windows iOS vs Android OpenAI vs Meta AI? Edit: Another observation is Meta seems to open things defensively. Facebook had a rich developer platform until Facebook stopped needing to grab market share. Meta’s VR platform was closed until Apple challenged them with Vision Pro. Then Meta announced open sourcing Horizon OS. I wonder if Meta will truly keep things open if they win, or if it’s more like a case of EEE?
- giancarlostoro 2y agoI've used OpenAI's tech, and I've used their competitors tech, but I've never used any of Facebooks AI tech yet. Is there something I'm missing out on?
- edm0nd 2y agoMeta = LLaMA I think the bet is that many other things will use their llama that you then use. Perhaps without you even knowing this.
- a_wild_dandan 2y agoRunning a (405B) flagship model locally on a Mac Studio? Otherwise, no. Most of these products are fairly similar.
- herval 2y agoEver heard of pytorch?
- giancarlostoro 2y agoNever used it myself, but that's a good call out and precisely why I asked.
- jsheard 2y agoWindows and Android took the commodity approach but they always had a business model, while Meta AI is currently running on Underpants Gnomes economics. 1. Spend billions on a product, then give it away for free. 2. ??? 3. Profit! > Then Meta announced open sourcing Horizon OS. Open sourcing isn't really the right term, they're allowing third party hardware vendors to use it but it's still proprietary. Horizon OS is built on top of Android and they're following the Android playbook where the core is technically open source but the version nearly everyone actually uses has a bunch of proprietary Google (or Meta) software layered on top, and Google (or Meta) dictates the terms of using that software, which lets them ensure that revenue always flows back to Google (or Meta) regardless of who made the hardware.
- Quekid5 2y ago"Open" is a meme at this point, isn't it?
- righthand 2y agoOpen as in you get to open the box. Or maybe Open as in “open for business”.
- epistasis 2y agoHaving used and learned from the architectures that Facebook has published via OCP over the past decade, this is not a "meme" but actual real information and designs that Facebook uses to commoditize their supply chain and that others can use too.
- throwaway48476 2y agoSome designs such as OCP NIC have become industry standard.
- Quekid5 2y ago... but that's been a thing since ever using the "Open" moniker? I just feel "Open" is a mental shortcut that's being misused a bit. See, e.g. [0] for an example of Facebook (now: Meta) for admirable dissemination of knowledge. [0] https://engineering.fb.com/2015/06/26/security/fighting-spam-with-haskell/ https://engineering.fb.com/2015/06/26/security/fighting-spam...
- epistasis 2y agoI've not noticed a change in the amount of usage of "open" or any mental shortcuts about it since it was introduced via the term "open source" in the 1990s. If anything, there's less discussion and abuse of the term now. The only memery that I see is from OpenAI, which had aspirations to openness in the past but which today is a sham. Facebooks open efforts here are completely great, IMHO, and have benefited me personally and professionally.
- deleted 2y ago[deleted]
- seydor 2y agoThey ve already gone after openAi, are they after Nvidia now?
- throwup238 2y agoNo, this is a rack built on NVIDIA’s platform, so this is just more $$$ for them.
- moffkalast 2y agoYeah this is... nothing. At least nothing anyone worth less than a few billion could ever care about. Would be far more interesting to see MTIA in an edge compute PCIe form.
- throwaway48476 2y agoThere's also an AMD rack and meta is big enough that they won't get blacklisted for it.
- mhandley 2y agoThe article talks about NVidia racks, but also about their DSF [0], which is Ethernet-based (as opposed to Infiniband) built on switches using Cisco and Broadcom chipsets and custom ASIC xPU accelerators such as their own MTIA [1] which is built for them by Broadcom. So they are talking more than one approach simultaneously. [0] https://engineering.fb.com/2024/10/15/data-infrastructure/open-future-networking-hardware-ai-ocp-2024-meta/ https://engineering.fb.com/2024/10/15/data-infrastructure/op... [1] https://ai.meta.com/blog/next-generation-meta-training-inference-accelerator-AI-MTIA/ https://ai.meta.com/blog/next-generation-meta-training-infer...
- KaoruAoiShiho 2y agoSort of, while yes this uses Nvidia, one of Nvidia's moats or big advantages is its rack scale integration. AMD and other providers just can't scale up easily, they are behind in terms of connecting tons of GPUs together effectively. So doing this part themselves instead of buying Nvidia's much-hyped (deservedly) NVL72 solution, which is a nonpareil rack system with 72 GPUs in it, and then opensourcing it, opens the door to possibly integrating AMD GPUs in the future and this hurts Nvidia's moat.
- Gee101 2y agoZuckerberg and Facebook gets a lot of hate but at least they invest a lot into engineering and open source.
- amelius 2y agoI personally think our academia should be training and curating these kinds of models and the data they are based on, but this is an acceptable second best.
- 123yawaworht456 2y agothere's already https://www.goody2.ai/chat https://www.goody2.ai/chat
- asdff 2y agoIMO there are much better ways to spend 300 million in research beyond firing up a cluster for 60 days to train on internet content.
- nickpsecurity 2y agoSpending $100 million one time on a GPT4-level model that is open-source would help with a lot of that research. Especially after all the 3rd party groups fine-tuned it or layered their tools on it. I think the Copilot-equivalent tools alone would make it quickly pay itself off in productivity gains. Research summaries, PDF extraction, and OCR would add more to that.
- mistrial9 2y agobasically no one in the entire world was willing to spend the kind of money on massive compute and data centers that Meta did spend, is spending and will spend. The actual numbers are (I think) rare to find and so large that it is hard to comprehend it.
- threeseed 2y agoAlso the fact that he is delivering on Fediverse integration with Threads. I don't think most people expected that to happen so quickly or frankly at all.
- m_ke 2y agoI wonder when Meta, Microsoft and OpenAI will partner on an open chip design to compete with NVIDIA. They’re all blowing billions of dollars on NVIDIA hardware with like 70% margin and with triton backing PyTorch it shouldn’t be that hard to move off of CUDA stack.
- deleted 2y ago[deleted]
- throwaway48476 2y agoEach of them is designing their own hardware. The goal isn't really to compete with nvidia though, whose market is general purpose GPU compute. Instead they're customizing hardware for inference to drive down product cost.
- infecto 2y agoWhat is a large mount of money to you, is not that significant to some of these companies. I suspect for the vast majority of these companies, it still represents a small expense. General sentiment is that there is probably overspending in the area but its better to spend it and not risk being left behind.
- m_ke 2y agoHalf of NVIDIA's 2nd quarter revenue (30 billion) came from 4 customers, with Microsoft and Meta already having spent 40-60 billion each on GPU data centers (of which most goes to NVIDIA). "Open"AI just raised a few billion and is supposedly planning on building their own training clusters soon. For a small fraction of that they could poach a ton of people from NVIDIA and publish a new open chip spec that anyone could manufacture. https://www.fool.com/investing/2024/09/12/46-nvidias-30-billion-revenue-4-mystery-customers/ https://www.fool.com/investing/2024/09/12/46-nvidias-30-bill...
- infecto 2y agoThat again underestimates the challenges in that undertaking. After all these costs are still drops on the bucket. Why distract yourself from your business to go and build chips. They all use SFDC, should they go and create and open source sales platform?
- TechDebtDevin 2y ago> "This effort pushed our infrastructure to operate across more than 16,000 NVIDIA H100 GPUs, making Llama 3.1 405B the first model in the Llama series to be trained at such a massive scale." So at 20k a pop (assuming meta has a decent wholesale price from Nividia) they spent $320 MILLION on the 405B model (not including probably 5-10 million in electricity for the training process, water, staff, infra). Do we think that brings more than 400+ million in value to Meta? I think so. I don't want to do the math, so I'll ask Perplexity to look it up: > "How much has Meta's valuation increased since they released their first open source model" Answer (edited): > Closing price on February 23, 2023: $509.50 > Closing price on October 11, 2024: $573.68 > The increase in stock price is $64.18 per share. > Total increase = Price increase per share × Number of outstanding shares > Total increase = $64.18 × 2,534,000,000 = $162,632,100,000 > Meta's stock valuation has increased by approximately $162.63 billion since the release of their first open source model on February 24, 2023. They seem to be making the right choices!
- JumpCrisscross 2y ago> Do we think that brings more than 400+ million in value to Meta? Tough to tell, given nobody is turning a net profit on LLMs yet. Companies have a tendency to develop neuroses, though, just like people. Apple’s near miss with bankruptcy fuelled cash hoarding. For Facebook, their disastrous IPO and near miss of mobile seems to have made them hyper aware of the Innovator’s Dilemma. $400mm spent on a defensive move is certainly wider than tens of billions on the metaverse.
- sdesol 2y ago> Tough to tell, given nobody is turning a net profit on LLMs yet. I suspect in the case of Meta and other big players, profit isn't necessary required to bring substantial value. Imagine their model being able to help them moderate more fairly and accurately. This alone could prevent potential legal actions from individuals, companies, and governments.
- JumpCrisscross 2y ago
- atomic128 2y agoAnd to power all those fused-multiply-add circuits, Meta will likely soon be involved in the construction or restart or life-extension of nuclear fission reactors, just like Microsoft and Google and Amazon (and Oracle, allegedly). Quoting Yann LeCun (Vice-President, Chief AI Scientist at Meta): AI datacenters will be built next to energy production sites that can produce gigawatt-scale, low-cost, low-emission electricity continuously. Basically, next to nuclear power plants. The advantage is that there is no need for expensive and wasteful long-distance distribution infrastructure. Note: Yes, solar and wind are nice and all, but they require lots of land and massive-scale energy storage systems for when there is too little sun and/or wind. Neither simple nor cheap. https://x.com/ylecun/status/1837875035270263014 https://x.com/ylecun/status/1837875035270263014
- gocdhkvsckdhnxv 2y ago[flagged]