8 ms·

Wow, 128 experts in a single model. That's a lot more than everyone else. The Snowflake team has a blog post explaining why they did that: https://www.snowflak

• throwup238 3y ago
  > Who's going to recoup all that investment? When? How? What's the long-term strategy AI of all these tech companies? Do they know something we don't? The first droid armies will rapidly recoup the cost when the final wars for world domination begin…
  • rvnx 3y ago
    Even before that, elections are coming end of the year, chat bots are great for telling whom to vote for. 2020's elections costed 15B USD in total, so we can't afford to lose (we are the good guys, right ?)
    • N0b8ez 3y ago
      How will the LLMs be used for this? They can't solve captchas, and they're not smart enough to navigate the internet by themselves. All they do is generate text.
      • kaibee 3y ago
        Transformers can definitely solve captchas. Not sure why you think otherwise.
        • N0b8ez 3y ago
          So captchas are obsolete now?
          • kaibee 3y ago
            For a while now, even before the latest AI models. Paid services exist (~2$/1k solves https://deathbycaptcha.com/ https://deathbycaptcha.com/)
• guessmyname 3y ago
  Why 0.6 million and not +600k ?
  • cs702 3y ago
    You're right. I changed it. Thanks!
• lewi 3y ago
  > over 0.6 million What a peculiar way to say: 600,000
  • cs702 3y ago
    You're right. I changed it. Thanks!
• cornholio 3y ago
  The model seems to be "build something fast, get users, engagement, and venture capital, hope you can grow fast enough to still be around after the Great AI cull". > offers over 0.6 million different pretrained open models. One estimate I saw was that training GPT3 released 500 tons of CO2 back in 2020. Out of those 600k models, at least hundreds are of a comparable complexity. I can only hope building large models does not become analogous to cryptocoin speculation, where resources are forever burned only in a quest to attract the greater fool. Those startups and researchers would better invest in smarter algorithms and approaches instead of trying to outpolute OpenAI, Meta and Microsoft.
  • cs702 3y ago
    > "build something fast, get users, engagement, and venture capital, hope you can grow fast enough to still be around after the Great AI cull" Snowflake is a publicly traded company with a market cap of $50B and $4B of cash in hand. It has no need for venture capital money. It looks like a case of "Look Ma! I can do it too!"
  • ReptileMan 3y ago
    >One estimate I saw was that training GPT3 released 500 tons of CO2 back in 2020 So absolute nothing in the grand scheme of things?
    • throwup238 3y ago
      Yeah that’s the annual emissions for only 100 people at the global average or about 30 Americans.
    • margalabargala 3y ago
      That's the amount that would be released by burning 50,000 gallons of gas, which is about that ten typical cars will burn throughout their entire lifespan. Done once, I agree, that's very little. But if each of those 600,000 other models used that much (or even a tenth that much), then that now becomes impactful. Releasing 500 tons of CO2 600,000 times over would amount to about 1% of all human global annual emissions.
• mlsu 3y ago
  Far fewer than 600,000 of those are pretrained. Most are finetuned which is much easier. You can finetune a 7B model on gamer cards. There is basically the big guys that everyone's heard of (google, meta, microsoft/openAI, and anthropic) and then a handful of smaller players who are training foundation models mostly so that they can prove to VCs that they are capable of doing so -- to acquire more funding/access to compute so that they may eventually dethrone openAI and take a piece of the multi-billion dollar "enterprise AI" market for themselves. Below that, there is a frothing ocean of mostly 7B finetunes created mostly by individuals who want to jailbreak base models for... reasons, plus the occasional research group. The most oddball one I have seen is the databricks LLM which seems to have been an exercise of pure marketing. Those I suspect will disappear when the bubble deflates a bit.
  • cs702 3y ago
    > an exercise of pure marketing Yes. Great choice of words. A lot of non-frontier models look like "an exercise of pure marketing" to me. Still, I fail to see the rationale for telling the world, "Look at us! We can do it too!"
    • mlsu 3y ago
      Mid-level managers at a lot of companies still have no clue what LLMs are or how they work. These companies (like databricks) want to have their salespeople upsell such companies on "business AI." They have the base model in their back pocket just in case one of the customers in the room has heard the name Andrej Karpathy before and starts asking questions about how good their AI solution is... they can point to their model and its benchmarks to say "we know what we are doing with this AI stuff." It's just standard marketing stuff which works right now because of how difficult it is to actually objectively benchmark LLMs.
  • theturtletalks 3y ago
    Yep, seems like every company is taking a longshot on a AI project. Even companies like Databricks (MosaicML) and Vercel (v0 and ai.sdk) are seeing if they can take a piece of this every growing pie. Snowflake and the like are training and releasing new models because they intend to integrate the AI into their existing product down the line. Why not use and fine-tune an existing model? Their in-grown model maybe better suited for their product. This can also fail like Bloomberg's financial model being inferior to GPT-4, but these companies have to try.
• TrueDuality 3y ago
  Most of those are fine tuned variants of open base models and shouldn't be included in the "every tech company" thing you're trying to communicate. Most of those are researcher or engineers learning how to work with these models, or are training them on specific data sets to improve their effectiveness in a particular task. These fine tunes are not a huge amount of compute, most of them are doing these trainings on a single personal machine over a day or so of effort, NOT the six+ months across a massive cluster it takes to make a good base model. That isn't wasted effort either. We need to know how to use these tools effectively, they're not going away. It's a very reductionist and inaccurate view of the world you're peddling in that comment.
• ganzuul 3y ago
  Money is for accounting. AI is a new accountant. Therefore money no longer is what it was.
• a13n 3y ago
  Seems like capitalism is doing its thing here. The potential future revenue from having the best model is presumably in the trillions.
  • bugbuddy 3y ago
    L0L Trillions ROFL
  • sangnoir 3y ago
    > The potential future revenue from having the best model is presumably in the trillions. I heard this winner-takes-all spiel before - only last time, it was about Uber or Tesla[1] robo-taxis making car ownership obsolete. Uber has since exited the self-driving business, Cruise is on hold/unwinding and the whole self-driving bubble has mostly deflated, and most of the startups are long gone, despite the billions invested in the self-driving space. Waymo is the only company with robo-taxis, albeit in only 2 tiny markets and many years away from general availability. 1. Tesla is making robo-taxi noises once more, and again, to juice investor sentiment.
    • a13n 3y ago
      Uber and Tesla are valued at 150B and 500B respectively, I'd say in terms of an ROI on deploying large amounts of capital these are both huge success stories. No investment in an emerging market is a sure thing, it's an educated guess. You have to take a lot of swings to occasionally hit a homerun, and investing in AI seems like the most plausible swing to make at this time.
      • sangnoir 3y ago
        I didn't claim there's no positive ROI. I only noted that the breathlessly promised "trillion+ dollar self-driving market" failed to materialize. I suspect the AI market will have a similar trajectory in the next decade: no actual AGI - maybe one company still plugging away at it, a couple of very successful companies whose core competencies don't include AI, but with billions in market cap, and a lot of failed startups littering the way there.
• analyte123 3y ago
  At a bare minimum, training and releasing a model like this builds critical skills in their engineering workforce that can't really be done any other way for now. It also requires compilation of a training dataset, which is not only another critical human skill, but also potentially a secret sauce if it turns out to give your model specific behaviors or skills. A big one is that it shows investors, partners, and future recruits that you are both willing and capable to work on frontier technology. Hard to put a price on this, but it is important. For the rest of us, it turns out you can use this bestiary of public models, mixing pieces of models with their own secret sauce together to create something superior than any of them [1]. [1] https://sakana.ai/evolutionary-model-merge/ https://sakana.ai/evolutionary-model-merge/
  • ankit219 3y ago
    These bigger companies are releasing open source models for publicity. For Databricks and Snowflake, both want enterprise customers, and want to show they can handle swathes of data and orchestration jobs, what better way to show that than by training a model. The pretraining part is done on a GPU but everything before that is managed on the Snowflake infra or Databricks. Databricks' website does focus heavily on this.[1] I am speculating here, they would use their own OSS models to create a proprietary version which does one thing well. Answering questions for customers based on their own data. It's not an easy problem to solve as it seemed initially given enterprises need high reliability. Need models which are good at tool use, and can be grounded well. They could have done it on an oss model, but only now we have Llama-3 which is trained to make tool use easy. (Tool use as in function calling and use of stuff like OpenAI's code interpreter) [1]: https://www.databricks.com/product/data-intelligence-platform https://www.databricks.com/product/data-intelligence-platfor...
• modeless 3y ago
  These projects all started a long time ago, I expect, and they're all finishing now. Now that there are so many models, people will hopefully change focus from training new duplicate language models to exploring more interesting things. Multimodal, memory, reasoning.
• jrm4 3y ago
  This seems to me to be the simple story of "capitalism, having learned from the past, undertands that free/open source is actually advantageous for the little guys." Which is to say, "everyone" knows that this stuff has a lot of potential. Everyone is also used to what often happens in tech, which is outrageous winner-take-all scale effects. Everyone ALSO knows that there's almost certainly little MARGINAL difference between what the big guys will be able to do and and what the little guys can do on their own ESPECIALLY if they essentially 'pool their knowledge.' So, I suppose it's the whole industry collectively and subconsciously preventing e.g. OpenAI/ChatGPT becoming the Microsoft of AI.
  • squigz 3y ago
    > This seems to me to be the simple story of "capitalism, having learned from the past, undertands that free/open source is actually advantageous for the little guys." This seems rather generous.
    • jrm4 3y ago
      Yeah, I don't mean it to sound that generous, as in "capitalism likes little guys." More like "little guys, or even literally any 'guy' that isn't dominant in this space -- which tends toward dominance -- have learned, perhaps counterintuitively, that free/open source is best for their own greedy interests." :)
• seydor 3y ago
  I am not worried. Someone will make a search engine to find the model that knows your answer . It will be called Altavista or Lycos or something
• barkingcat 3y ago
  Training LLM is the cryptobro pivot.
• DowagerDave 3y ago
  Snowflake has a pretty good story in this space: "Your data is already in our cloud, so governance and use is a solved problem. Now use our AI (and burn credits)". This is a huge pain-point if you're thinking about ML with your (probably private) data. It's less clear if this entices companies to move INTO Snowflake IMO And streamlit, if you're as old as me, looks an awful lot like a MS-Access application for today. Again, it lives in the database, runs on a Snowflake warehouse and consumes credits, which is their revenue engine.
  • hnthrowaway9812 3y ago
    Snowflake could have the same story by hosting Llama 3 which is probably more efficient/better.
    • throwaway120487 3y ago
      Snowflake hosts a couple models: https://docs.snowflake.com/en/user-guide/snowflake-cortex/llm-functions#availability https://docs.snowflake.com/en/user-guide/snowflake-cortex/ll...
• Onavo 3y ago
  > Who's going to recoup all that investment? When? How? Hype and jumping on the bandwagon are perfectly good reasons for a business. There's no business without risk. This is the cost of doing business which you want to explore greenfield projects.
• _flux 3y ago
  And huggingface is hosting (randomly assuming 8-64 GB per model) 5..40 PB of models for free? That's generous of them. Or can the models share data? Ollama seems to have some ability to do that.
• temuze 3y ago
  In the short-term, these kinds of investments can hype up a stock and create a small bump. However, in the long-term, as the hype dies down, so will the stock prices. At the end of the day, I think it will be a transfer of wealth from shareholders to Nvidia and power companies.
  • peteradio 3y ago
    > as the hype dies down, so will the stock prices. *Depending on govt interventions
  • LordDragonfang 3y ago
    I just wish that AMD (and, pie in the sky, Intel) had gotten their shit together enough that these flaming dumptrucks full of money would have actually resulted in a competitive GPU market. Honestly, Zuckerburg (seemingly the only CEO willing to actually invest in an open AI ecosystem for the obvious benefits it brings them) should just invest a few million into hiring a few real firmware hackers to port all the ML CUDA code into an agnostic layer that AMD can build to.
    • hnthrowaway9812 3y ago
      Groq seems to be well positioned to give Nvidia a run for their money, actually.
• blackeyeblitzar 3y ago
  > What's the rationale for releasing all these models to the public? Do all these tech companies know something we don't? Why are they doing this? It’s mostly marketing for the company to appear to be modern. If you aren’t differentiated and if LLMs aren’t core to your business model, then there’s no loss from releasing weights. In other cases it is commoditizing something that would otherwise be valuable for competitors. But most of those 600K models aren’t high performers and don’t have large training budgets, and aren’t part of the “race”.
• richardw 3y ago
  It diminishes the story that Databricks is the default route to privately trained models on your own data. Databricks jumped on the LLM bandwagon really quickly to good effect. Now every enterprise must at least consider Snowflake, and especially their existing clients who need to defend decisions to board members. It also means they build large scale rails necessary to use Snowflake for training and can market such at every release.
  • BPA0 2y ago
    I don't think that you understand Databricks. Databricks gives you the tools to train, tune or build RAG models. Snowflake doesn't. Having said that, I'm a big fan of Llama-3 at the moment.
    • buttaphingas 2y ago
      Snowflake has some examples of that: https://www.snowflake.com/blog/easy-secure-llm-inference-retrieval-augmented-generation-rag-cortex/ https://www.snowflake.com/blog/easy-secure-llm-inference-ret... https://quickstarts.snowflake.com/guide/asking_questions_to_your_own_documents_with_snowflake_cortex/index.html#0 https://quickstarts.snowflake.com/guide/asking_questions_to_... And I assume these new models would be now available for embedding. Is this different to what Databricks offers?