4 ms·
As the article suggests, the presence of LLAMA is decreasing demand for GPUs. Which are critical to Metas ad recommendation services. Ironically, by supporting
by vineyardmike 2y ago
As the article suggests, the presence of LLAMA is decreasing demand for GPUs. Which are critical to Metas ad recommendation services.
Ironically, by supporting the LLM community with free compute-intense models, they’re decreasing demand (and price) for the compute.
I suspect they’ll never directly monetize LLAMA as a public service.
- jorvi 2y ago> Ironically, by supporting the LLM community with free compute-intense models, they’re decreasing demand (and price) for the compute. For other people that that sentence didn't make sense for at first glance: "by supporting the LLM community with free compute-intense models [to run on their own hardware] they’re decreasing demand (and price) for the compute [server supply]."
- fragmede 2y agoHow fungible is that compute though? Having even a single H100 is different than having a bunch of 4090's, nevermind a properly networked supercomputer of H100s.
- vineyardmike 2y agoThat’s the point. You can run inference on a 4090 but training is better on a H100. If you use llama, you don’t need to train on an H100, so you can free that supply up for meta.
- fragmede 2y agoI haven't been following llama closely but I thought the latest model was too big for inference on 4090's, and that you can't fine tune on 4090's either, but furthermore, the other question is if the market is there for running inference on 4090s.
- vineyardmike 2y agoWell, (1) there are a ton of GPUs out there of various specs, and you can also use an inference provider who can use a H100 or similar to serve multiple inference requests at once. (2) there are a ton of LLAMA sizes, from 1b, 2b, 8b, 70b, and 400b. The smaller ones can even run on phone GPUs.
- vineyardmike 2y agoSorry, I should have been more clear. They’re decreasing demand for expensive GPUs that would be required to train a model. Fine-tuning and inference are less compute intense, so overall demand for top-end GPU performance is decreased even if inference compute demand is increased. Basically, why train an LLM from scratch, and spend millions on GPUs, when you can fine tune LLAMA and spend hundreds instead.
- jorvi 2y agoThank you for the extra clarification, I hadn’t even thought of inference vs training!
- grues-dinner 2y agoWith all these billions upon billions in AI hardware screaming along, are ads actually that much better targeted than they used to be? I imagine admongers like Meta and Google have data that shows they are right to think they have a winning ticket in their AI behemoths, but if my YouTube could present any less relevant ads to me, I'd be actually impressed. They're intrusive, but actually they're so irrelevant that I can't even be bothered to block them, because I'm not going to start online gambling or order takeaways.
- vineyardmike 2y agoA better question, with a growing push for privacy, how can they keep ads from regressing? There’s a lot more that goes into the ad space than just picking which ad to show you, and it’ obviously depends on who wants to reach you. For example, probabilistic attribution is an important component on confirming that you actually got the ad and took the action across multiple systems. Also, since you mentioned it, TV ads tend to be less targeted because they’re not direct-action ads. Direct action ads exist in a medium where you can interact with the ad immediately. Those ads are targeted to you more, because they’re about getting you to click immediately. TV ads are more about brand recognition or awareness. It’s about understanding the demographic who watches the show, and showing general ads to that group. Throw a little tracking in there for good measure, but it’s generally about reaching a large group of people with a common message.
- mark_l_watson 2y agoYou ask a great question, and I wonder how the push for more privacy will pan out (pardon the gold mining analogy). I am almost done with the very good new book The Tech Coup by Marietje Schaake, and I have also read Privacy is Power and Surveilance Capitalism. I think more of the public is waking up to the benefits of privacy. All that said, I am an enthusiastic paying customer of YouTube Prime and Music, Colab (I love Colab), and sometimes GCP. For many years I have happily have told Google my music and YouTube preferences for content. I like to ask myself what I am getting for giving up privacy in a hopefully targeted and controlled way.