9 ms·

Databricks Strikes $1.3B Deal for Generative AI Startup MosaicML

188 points by jmsflknr 3y ago

mikpanko 3y ago
https://archive.ph/eTGvR https://archive.ph/eTGvR
stingraycharles 3y ago
I can’t help but feel this is mostly hype driven by a company that’s looking to reinvent themselves while facing the prospect of irrelevance. The piece reads like it’s a press release, with abstract things such as “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs, as the current AI “hype” cycle has been going on for almost a decade already. Corporate leaders have always been under pressure to get their data cleaned and in the correct shape, and Databricks’ business model is to sell to these corporate leaders expensive packages combined with consultancy with generally very high operational costs. Compared to 5 years ago, nowadays I rarely encounter people that actually care about Databricks’ product offering.
- anonylizard 3y ago
  A rising tide lifts all boats. Databricks may have its thunder stolen by snowflake. But the true AI boom happening right now, benefits many data product vendors. The basic requirement for enterprises to use LLMs is having their data in order, which basically requires a cloud data warehouse. It is simply responsible to profit off the hype for a established data company.
  - DebtDeflation 3y ago
    > The basic requirement for enterprises to use LLMs is having their data in order, which basically requires a cloud data warehouse. I'm curious why you would say this. The main use cases for LLMs involve things like customer service chatbots, knowledgebase search, document summarization, co-pilot/code generation, and content generation for product descriptions and marketing emails. The main data sources would be things like document repositories on Sharepoint and transcripts of customer calls. Not a bunch of historical sales data or financial data sitting in the typical data warehouse. I think there's a big misconception about this. Data sources for Generative AI are very different than data sources for 2013 era data science projects (which perhaps not coincidentally is what led to the development of things like Databricks).
    - deleted 3y ago
      [deleted]
- kadomony 3y ago
  As someone new to the data ops platform world.. what does Databricks even do? They get so much hype as a great employer, but why are they attracting people if their secret sauce is irrelevant?
  - moneywoes 3y ago
    Further, what is it different from snowflake that they do?
    - pram 3y ago
      The biggest difference to consider is storage, I guess. Databricks clusters can store in S3 or wherever you want really. Snowflake has their own storage layer and it always lives in their cloud. This precludes any on-prem deployments.
      - ramraj07 3y ago
        You can’t be more wrong? Snowflake also stores in s3 or elsewhere, in fact it’s like the only common feature. Snowflake uses proprietary formats mostly though and only gives a sql interface. Think of it as Apple vs android.
        cfeduke 3y ago
        > You can’t be more wrong? Snowflake also stores in s3 or elsewhere Snowflake stores data in S3 in Snowflake's various AWS accounts. Egress fees are necessary when performing anything outside of Snowflake. Databricks operates on data stored in your S3 on your AWS account(s). Databricks also runs on your compute contained within your AWS account(s). Both approaches have valid use cases.
        chrisjc 3y ago
        > Snowflake stores data in S3 in Snowflake's various AWS accounts. Egress fees are necessary when performing anything outside of Snowflake. Egress fees are only if you are unloading into a different region/cloud. If your data is in SF AWS us-east-1, and you unload to your S3 in us-east-1, there is no egress fee. > Snowflake charges a per-byte fee for data egress when users transfer data from a Snowflake account into a different region on the same cloud platform or into a completely different cloud platform. Data transfers within the same region are free. https://docs.snowflake.com/en/user-guide/cost-understanding-data-transfer https://docs.snowflake.com/en/user-guide/cost-understanding-...
- throwaway54232 3y ago
  This was my guess as well. I am wondering what you see as the "prospect of irrelevance" they are facing. I have my guess but I would like to hear your take.
  - stingraycharles 3y ago
    People are doing more and more self-hosting and desire PaaS-like offerings that run in their own cloud, rather than SaaS “we manage your spark cluster for you” which is what DataBricks was founded upon. PaaS has significantly thinner margins, but it’s desirable from the customers’ perspective as it’s cheaper and much better from a (data) security point of view. As such, they’ll be forced to do a lot more services oriented work rather than product / platform oriented work, because it pays well. Their sales team is also excellent. I see a similar fate for them as Cloudera.
    - kthejoker2 3y ago
      > People are doing more and more self-hosting and desire PaaS-like offerings that run in their own cloud, rather than SaaS Do you have some data to support this? This is a pretty bold claim.
      - hodgesrm 3y ago
        +1. I would be delighted to see that data as well. I mean we all like the idea of local control but local PaaS is hard to do in a general way.
    - bobbruno 3y ago
      You can find an article on Bloomberg stating that Databricks’ annual revenue has grown by 60%, which sort of backs the SaaS hypothesis it bets on. Do you have data on your PaaS hypothesis that is bigger than that? Source: https://www.bloomberg.com/news/articles/2023-06-13/databricks-hits-1-billion-in-annual-sales-while-adding-data-warehouse-tool https://www.bloomberg.com/news/articles/2023-06-13/databrick...
- spmurrayzzz 3y ago
  I don't know if I'd say this is reinventing themselves, or at least not from the ground up, they've been somewhat active in the open source LLM scene for some time now. Not that releasing dolly alone caused some major foundational shift, but it was a solid contribution. Additionally, acquiring Mosaic fills a pretty gaping hole in their existing product offering. The whole process of pretraining, fine-tuning, and evaluating open source LLMs in production is a total nightmare currently. Mosaic's stack at least attempts to make that better (on top of releasing solid foundational models themselves). > “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs I agree that its buzzwordy and a little abstract. But also in my experience, getting "data ready for AI" is actually the primary constraint many orgs have with respect to using LLMs in an enterprise context. Their data is not stored in a way thats easy to tokenize/label/embed for effective training or fine-tuning. And you could argue the preprocessing actually should the easy part for non-ML devs to tackle, as its primarily a software engineering problem. Yet, its still the thing that keeps many folks from getting started (before you even get to attempt to tackle the inference problem).
  - ethbr0 3y ago
    From my understanding, Databrick's core value proposition is -- 'We let you do things you want to, with the people you have.' It may surprise HN to learn, but most companies don't have top-tier technical talent. Consequently, how do you do {cutting edge thing board and investors are demanding company do} with the people you have/can afford? Use an offering that decreases the necessary skill level by providing powerful prebuilt components. From this perspective, LLM integration sounds like a perfect fit. It's something every company is being asked to have a plan on, but one few are technically staffed to execute on their own.
    - stakhanov 3y ago
      > [...] LLM integration [...] is something every company is being asked to have a plan on, but one few are technically staffed to execute on their own. But this is still the "language of hype". LLM integration to do what? I'm not saying they're not doing anything useful, just that this is not a way to describe either a productivity-enhancing technology (feature) or an actual value proposition (benefit) in a meaningful way.
- pletnes 3y ago
  What has replaced Databricks, in your opinion? Doesn’t seem «dead» to me, yet. Serious question.
  - aynyc 3y ago
    Their moat was to be able to spin up spark clusters for you. This was very useful when companies just starting in that space and don’t have the skills. But with K8s and clusters becoming more common for normal workload, it’s not hard to manage your own spark clusters anymore. There goes their moat. Their other offerings just aren’t that amazing. People now don’t just want to buy hand tools anymore, they want power tools. AI/ML and its cousins are in the power tool space. Databrick cost is stupid high, like snowflake. Any companies looking to chop budget would put those contracts up first.
    - disgruntledphd2 3y ago
      Their moat is their phenomenal sales team. Like, I'm at company 3 that pays for databricks. Amount of those companies that use Spark = 0. I've given up complaining now.
      - esafak 3y ago
        So what are they using, the notebooks?
        pletnes 3y ago
        But in the notebooks you run pyspark / scala spark / spark sql code?
        esafak 3y ago
        Yes, you can, if you want.
        disgruntledphd2 3y ago
        Yes, but for most of these companies they don't actually run (much or any) spark stuff, they just use the notebooks. It's really depressing, tbh but I've made my peace with it now.
- quickthrower2 3y ago
  Looking at the Mosaic website, and looking at many AI startups, I can't help feeling like they are all small AWS Sagemaker, Bedrock, Trainium, ... (etc.) competitors, and not sure how they will compete with the sheer capital that Amazon has, and potential to offer that compute power cheaper (at least long enough to kill off most of the competition). Maybe it is it the off chance that one of these companies might become a big player worth billions that makes the odds worth it?
  - george_123 3y ago
    A lot of AWS services (especially SageMaker) aren’t very good in customer experience. People buy them for nominal capabilities and AWS core bread and butter — short-term and long-term reliability. Most of these startups (AI and others) have to offer a compelling product before even being notable. Besides, AWS top level doesn’t care if you use sagemaker or not. There’s a premium but if you’re still using EC2 via another startup, they’re still capturing lions share of value.
    - dbish 3y ago
      Sagemaker is probably a bad example since most folks who run ML workloads on AWS don't use it (from what I've seen talking to many ML teams). It's partially a scattered focus on their customer type (are they for experts/non-experts/something in between?, what exact use cases are they covering?) and partially I think just bad UI/UX (related to customer focus). AWS will converge on what is working for ML platforms and just build their own as time goes on. First mover advantage won't matter and these earlier ML platforms will be consumed imho
  - tim_sw 3y ago
    The devil is in the details. Training large LLMs requires a lot of custom infra (handling GPUs going down, efficiently pushing data to keep the accelators busy, deciding on which mechanism of parallelizing model training is better - data vs model parallelism or both, tuning hyperparams of optimizers which can be different for larger batch sizes, etc) Mosaic is one of the better providers for this. AWS is nowhere near ready at this current point in time, it is pretty much a "dumb" infra provider in large LLM training at this point. (Of course they won't be standing still and will prob acquire that capability one way or another)
- fakedang 3y ago
  Just curious, how is Databricks going to be irrelevant? I thought they were a reasonably powerful database provider for data analytics platforms in many companies?
  - millenseed 3y ago
    Yeah that's an absolutely garbage idea. Databricks is basically the opposite of irrelevant at this point.
paulsutter 3y ago
This $1.3B dollar figure is surely relative to Databricks 2021 valuation from their Series H (most recent round), and doesn’t represent real money since multiples have plunged since then Estimates of Databricks revenue is $400M and “valuation” of $36B, which would never hold today
- _bohm 3y ago
  Where do you get the $400m figure? Just a couple weeks ago Bloomberg ran an article stating their revenue is >$1B
  - noirbot 3y ago
    Yea, with what they charge, I'd be surprised if it was as low as 400 mil. I've seen mid-sized companies with 8-figure Databricks contracts.
- tedivm 3y ago
  Databricks wasn't the one purchased, MosaicML was. Mosaic's last round had a valuation of $136m, so this is a pretty big jump for them.
  - gk1 3y ago
    The Databricks valuation is relevant if any part of the acquisition was paid for in Databricks equity.
    - jamesblonde 3y ago
      Any part? My guess is 100% of it.
- brentm 3y ago
  I heard Ali the CEO of Databricks speaking on a podcast recently and stated their revenue last year was north of 1B.
htrp 3y ago
Turns out Databricks couldn't do it all internally with just training the dolly model. I think it's a good accretive deal because Databricks already has a good number of enterprise relationships and Mosaics key offering has been training software with their own optimizations that increase gpu efficiency.
cheptsov 3y ago
Congrats to the MosaicML and wish all the luck! If anyone is interested in MosaicML alternatives, check out dstack, we are building its OSS and cloud-agnostic alternative: https://dstack.ai https://dstack.ai. Disclaimer: I'm the founder and CEO at dstack.
- huac 3y ago
  It would be really cool to support more vendors (eg Lambda Labs, Runpod, etc) - getting GPU availability feels like 'any port in a storm' sometimes :)
  - cheptsov 3y ago
    Yup, we're already working on an experimental support for Lambda Labs! Ping us if you'd like to test it out. Perhaps we could show something already next week.
jgalt212 3y ago
> bringing down the cost of using generative AI—from tens of millions of dollars to hundreds of thousands of dollars per model, That's a lot of people you have to replace with one model--even if you get the cost basis down. Do those costs include the period refitting?
- Tostino 3y ago
  It really depends on what you are using it for. There are plenty of high-value use cases where you can free up resources that are spending time inefficiently due to handling tasks that a domain specific LLM could help with. Maybe that allows you to not expand your team by allowing your existing resources to work on things that bring more value.
poomer 3y ago
This class of startup, "build domain specific LLMS using your own data", is extremely crowded right now but I am not optimistic about their future. For large companies, the actual modeling work for this is already easy for any ML team, thanks to existing FOSS work on stuff like PEFT and LoRA. The hard part is figuring out what data goes into the fine tuning process and how to get this data in a usable form, but this is very business specific and can't be automated in a SaaS process. For SMBs, the value would be in using the LLM to generate responses to customer Q&A/search queries. But these companies aren't going to integrate some external third party service, they'll only use it if it's already baked into their CMS - Wordpress/Shopify/Wix/etc. I just don't see who the final consumer for this product would be.
- re-thc 3y ago
  > is extremely crowded right now but I am not optimistic about their future Why not? Every larger "cloud" company seems to be randomly buying 1 at the moment to offer "AI" so might get some good deals. This is clearly 1 of them - panic buy.
- brucethemoose2 3y ago
  > thanks to existing FOSS work on stuff like PEFT and LoRA YMMV. Sometimes a LORA is fine, but sometimes a full finetune is necessary for higher quality output. That being said, backwards pass free training keeps making more and more progress. Seems like a short matter of time before it becomes practical.
  - Tostino 3y ago
    Look at QLoRA. The QLoRA can be attached to all layers, allowing you to alter behavior with much less data than the original LoRA implementation. It seems to "stick" better. I just fine tuned a ~30b parameter model on my 2x 3090s to check it out. It worked fantastically. I should be able to fine tune up-to 65b parameter models locally but wanted to get my dataset right on a smaller model before trying.
    - throwaway72762 3y ago
bratao 3y ago
Thrilled to hear about MosaicML's successful exit! Nonetheless, this could serve as an indication to explore other options. Given Databricks' past acquisition of Redash which led to its downfall, there's no absolute assurance that Mosaic's fate won't be similar.
- orderedtree 3y ago
  Why do you say it led to redash's downfall? What happened to redash since the acquisition?
  - bratao 3y ago
    They've essentially killed Redash, which was one of the best open-source dashboard/data visualization tools. They had promised to keep it open-source and improve upon it (https://web.archive.org/web/20211019150919/https://blog.redash.io/redash-joins-databricks/ https://web.archive.org/web/20211019150919/https://blog.reda...). However, a year later, their SaaS has been discontinued. The open-source repository is now stagnant, with hundreds of unresolved merge requests. On a positive note, they recently shared the repository with some other open-source maintainers, so there's hope that Redash will be reborn.
    - justinclift 3y ago
      > On a positive note, they recently shared the repository with some other open-source maintainers, so there's hope that Redash will be reborn. Yeah, the root of the problem (in my opinion) was that Redash has sooo many Python dependencies (due to supporting so many databases and similar) that it's become a real hairball source code wise to keep them all playing together nicely. Especially as over time (say) library package FOO has some security vulnerability reported that gets fixed in a new release... but the dependencies of the newer release are too new to work with (say) package BAR. Times that by 50 and it's a real pita. Simultaneously to that, the Redash team got busy with their work at Databricks (mostly not Redash related). Then the automatic CircleCI checks on PRs started failing (ugh), etc. --- But, as @bratao mentioned above that's all getting worked through now. Admin and maintainer permissions have been given to a group of dev volunteers / known Redash enthusiasts. CI is working again now (as of last night), and we're currently untangling the dependency hairball. It's likely to be a few weeks (minimum I guess) before any new official releases are ready, but it will happen. :)
ShamelessC 3y ago
What do these companies do? I consider myself mildly well versed in the ML space at least, but I’ve never heard of MosaicML before (have they trained anything popular?). I’ve heard of Databricks but know very little about what service they provided before/after the recent deep learning craze.
- Zizizizz 3y ago
  Databricks provides Jupyter lab like notebooks for analysis and ETL pipelines using spark through pyspark, sparkql or scala. I think R is supported as well but it doesn't interop as well with their newer features as well as python and SQL do. It interfaces with cloud storage backend like S3 and offers some improvements to the parquet format of data querying that allows for updating, ordering and merged through https://delta.io https://delta.io . They integrate pretty seamlessly to other data visualisation tooling if you want to use it for that but their built in graphs are fine for most cases. They also have ML on rails type through menus and models if I recall but I typically don't use it for that. I've typically used it for ETL or ELT type workflows for data that's too big or isn't stored in a database.
  - ShamelessC 3y ago
    Thanks!
  - tsunamifury 3y ago
    Now say it even dumber as if I were a dog or a ceo.
    - quickthrower2 3y ago
      Perhaps, it is a fancy .csv importer?
    - esafak 3y ago
      They create services for data scientists (notebooks) and machine learning engineers (Spark).
underlines 3y ago
Coincidentally this comes after MosaicML released the best open source commercially usable LLMs on huggingface: mpt-30b, the first open source LLM with 8k context length that can be extended even further with ALiBi and has been trained on a whopping 1 trillion tokens vs. 300 billion for Pythia and OpenLLaMA, and 800 billion for StableLM.
- jncraton 3y ago
  OpenLLaMA models up to 13B parameters have now been trained on 1T tokens: https://github.com/openlm-research/open_llama https://github.com/openlm-research/open_llama
  - underlines 3y ago
    unfortunately not openllama-33b yet
    - emadm 3y ago
      20b done
- brucethemoose2 3y ago
  Is it better than Falcon 40B?
- swyx 3y ago
  do we know how it ranks vs the other models yet? its not yet up on https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
brucethemoose2 3y ago
Falcon 7B/40B is good, but hasn't really caught on like LLaMA has. One big reason is that its not well supported by 4-bit inference code (namely llama.cpp and GPTQ). Mosaic's 7B (and 30B?) models have the same issue, and 7B kinda paled in comparison to LLaMA 7B... But maybe it would be better if finetuned? To me, its kinda baffling that Mosaic didn't work on adding highly quantized inference to the popular frameworks.
- Tostino 3y ago
  Falcon is also slow for inference compared to LLaMA models of similar size. Speed can turn into a quality all its own when you break up problems into small pieces and use the LLM to iterate.
  - brucethemoose2 3y ago
    Thats because its unoptimized, right? My impression is that the Falcon GPTQ (and 8 bit bnb inference?) code is just immature.
- ml_hardware 3y ago
  Mosaic's MPT models are already supported in GGML: https://github.com/ggerganov/ggml https://github.com/ggerganov/ggml Here's MPT-30B running in 4-bit precision on CPU :) https://twitter.com/abacaj/status/1673133443339763712?s=20 https://twitter.com/abacaj/status/1673133443339763712?s=20
  - brucethemoose2 3y ago
    Oh I didn't realize this. Everything is moving so fast that I can't even keep up with the features.
sqs 3y ago
Congratulations to folks at MosaicML and Databricks!
YChacker100 3y ago
Congratulations to all involved!!
woeirua 3y ago
How old was MosaicML? How many customers did they have? Revenue? Profits? This feels like pure hype for LLMs to be honest. Good for the founders of MosaicML to exit at the top, but I'd bet good money that DataBricks winds up writing this off in a year or two max.
swyx 3y ago
huge congrats to the team - we interviewed Jonathan and Abhi from the team last month ( https://www.latent.space/p/mosaic-mpt-7b#details https://www.latent.space/p/mosaic-mpt-7b#details ) and it was blindingly obvious that they were an incredible team of engineers building one of the most valuable training platforms in the industry. congrats!
- CrimsonWanderer 3y ago
  Good to hear they are responsive to PR opportunities. Too bad their interview process doesn't match this level of enthusiasm. I had a recruiter reach out from DB late last year. I wasn't looking for new work but it seemed like it'd be worth a chat. I had to move my schedule around to fit it in and then get up extra early to be prepared for the chat...only the recruiter never showed. Didn't follow up about missing a meeting they had scheduled. It's been crickets. That is enough of a lack of professionalism really stood out as a red flag. Hard pass.
jedberg 3y ago
I didn't realize Databricks was doing so well they could spare $1B in equity/cash. Congrats not only to MosaicML but Databricks as well!
- hackitup7 3y ago
  Yeah, they're one of the strongest private software companies right now. I'd say easily top 10.
  - dbish 3y ago
    It's so suprising to me that there's enough dropped on the floor/use cases ignored from AWS/Azure that they could build such a large business.
cavisne 3y ago
Wonder what the outcome for early employees looks like here? This is not a very big company and hasn't been around for very long. 1.3B is a lot of money
ttymck 3y ago
~50 employees? I'd love to hear what regular employees will see from this (if anything). Even if it's completely controlled by NDA, I think even confirming the existence of that NDA would be helpful. So often we see these "success" stories and we don't hear exactly how the "regular joes" made out. And it can color our perception of the startup gamble, without real evidence.
DropInIn 3y ago
In light of the story I see just above this one ( https://news.ycombinator.com/item?id=36475376 https://news.ycombinator.com/item?id=36475376 ) Imma call inflated bs....
OthmaneHamzaoui 3y ago
I'm less surprised by the acquisition but more confused by seeing MosaicML labeled as 'Generative AI startup'. To my knowledge MosaicML is an open source lib for optimising training/inference taking a "speedup" tricks from research papers and making them available easily. At least a few months back that was the case. When did they become a Gen AI startup ? Or they rebranded lately?
zht 3y ago
Unless if this 1.3B cash I wouldn’t celebrate so soon. This is likely mostly stock deal, backed by databricks equity that, like most other private companies, is illiquid and has taken a nose dive in value in the last couple of years. I don’t know the exact split, but I’d imagine it is 300m cash, 1B in databricks stock at some last round private valuation that’s likely untested Databricks last raises at 38B valuation at 1B ARR near peak valuation hype in August 2021. Assuming they can get 10x and their ARR has doubled since then, they’re down maybe 50 percent in valuation. Internally they say they are marking themselves down to about 34B. Is the deal using databricks shares valued at 38B? 34B? 20B? Who knows. I hope it’s the latter. And I hope that whenever Mosaic folks get to sell their databricks stocks that it is worth more than that
j_shi 3y ago
Actually seems Databricks got a great deal for Mosaic. Real qustion is why Mosaic took it v. hold out or do another round Rough math plugging in public #s and comments here: - All stock deal at Aug 2021 val of 38B (1B ARR) - Assume rev doubled to 2B (which may even be aggressive) - SAAS multiples are down 6x since Aug 2021 - 38B x 2 / 6 = $12.7B - 12.7B / 38B * 1.3B = 434M = effective price - Assume 100M to pref stock --> Comes out to 334M, with a chunk of that (1/3? 1/4?) potentially subject to earn out
- j_shi 3y ago
  Realized there's probably pref on Databricks too, which would further lower the value of its common. On the other hand, there could have been a markdown from the 38B since August '21

188 points by jmsflknr 3y ago

https://archive.ph/eTGvR https://archive.ph/eTGvR

I can’t help but feel this is mostly hype driven by a company that’s looking to reinvent themselves while facing the prospect of irrelevance. The piece reads like it’s a press release, with abstract things such as “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs, as the current AI “hype” cycle has been going on for almost a decade already. Corporate leaders have always been under pressure to get their data cleaned and in the correct shape, and Databricks’ business model is to sell to these corporate leaders expensive packages combined with consultancy with generally very high operational costs. Compared to 5 years ago, nowadays I rarely encounter people that actually care about Databricks’ product offering.

anonylizard 3y ago
A rising tide lifts all boats. Databricks may have its thunder stolen by snowflake. But the true AI boom happening right now, benefits many data product vendors. The basic requirement for enterprises to use LLMs is having their data in order, which basically requires a cloud data warehouse. It is simply responsible to profit off the hype for a established data company.
- DebtDeflation 3y ago
  > The basic requirement for enterprises to use LLMs is having their data in order, which basically requires a cloud data warehouse. I'm curious why you would say this. The main use cases for LLMs involve things like customer service chatbots, knowledgebase search, document summarization, co-pilot/code generation, and content generation for product descriptions and marketing emails. The main data sources would be things like document repositories on Sharepoint and transcripts of customer calls. Not a bunch of historical sales data or financial data sitting in the typical data warehouse. I think there's a big misconception about this. Data sources for Generative AI are very different than data sources for 2013 era data science projects (which perhaps not coincidentally is what led to the development of things like Databricks).
  - deleted 3y ago
    [deleted]
kadomony 3y ago
As someone new to the data ops platform world.. what does Databricks even do? They get so much hype as a great employer, but why are they attracting people if their secret sauce is irrelevant?
- moneywoes 3y ago
  Further, what is it different from snowflake that they do?
  - pram 3y ago
    The biggest difference to consider is storage, I guess. Databricks clusters can store in S3 or wherever you want really. Snowflake has their own storage layer and it always lives in their cloud. This precludes any on-prem deployments.
    - ramraj07 3y ago
      You can’t be more wrong? Snowflake also stores in s3 or elsewhere, in fact it’s like the only common feature. Snowflake uses proprietary formats mostly though and only gives a sql interface. Think of it as Apple vs android.
      - cfeduke 3y ago
        > You can’t be more wrong? Snowflake also stores in s3 or elsewhere Snowflake stores data in S3 in Snowflake's various AWS accounts. Egress fees are necessary when performing anything outside of Snowflake. Databricks operates on data stored in your S3 on your AWS account(s). Databricks also runs on your compute contained within your AWS account(s). Both approaches have valid use cases.
        chrisjc 3y ago
        > Snowflake stores data in S3 in Snowflake's various AWS accounts. Egress fees are necessary when performing anything outside of Snowflake. Egress fees are only if you are unloading into a different region/cloud. If your data is in SF AWS us-east-1, and you unload to your S3 in us-east-1, there is no egress fee. > Snowflake charges a per-byte fee for data egress when users transfer data from a Snowflake account into a different region on the same cloud platform or into a completely different cloud platform. Data transfers within the same region are free. https://docs.snowflake.com/en/user-guide/cost-understanding-data-transfer https://docs.snowflake.com/en/user-guide/cost-understanding-...
throwaway54232 3y ago
This was my guess as well. I am wondering what you see as the "prospect of irrelevance" they are facing. I have my guess but I would like to hear your take.
- stingraycharles 3y ago
  People are doing more and more self-hosting and desire PaaS-like offerings that run in their own cloud, rather than SaaS “we manage your spark cluster for you” which is what DataBricks was founded upon. PaaS has significantly thinner margins, but it’s desirable from the customers’ perspective as it’s cheaper and much better from a (data) security point of view. As such, they’ll be forced to do a lot more services oriented work rather than product / platform oriented work, because it pays well. Their sales team is also excellent. I see a similar fate for them as Cloudera.
  - kthejoker2 3y ago
    > People are doing more and more self-hosting and desire PaaS-like offerings that run in their own cloud, rather than SaaS Do you have some data to support this? This is a pretty bold claim.
    - hodgesrm 3y ago
      +1. I would be delighted to see that data as well. I mean we all like the idea of local control but local PaaS is hard to do in a general way.
  - bobbruno 3y ago
    You can find an article on Bloomberg stating that Databricks’ annual revenue has grown by 60%, which sort of backs the SaaS hypothesis it bets on. Do you have data on your PaaS hypothesis that is bigger than that? Source: https://www.bloomberg.com/news/articles/2023-06-13/databricks-hits-1-billion-in-annual-sales-while-adding-data-warehouse-tool https://www.bloomberg.com/news/articles/2023-06-13/databrick...
spmurrayzzz 3y ago
I don't know if I'd say this is reinventing themselves, or at least not from the ground up, they've been somewhat active in the open source LLM scene for some time now. Not that releasing dolly alone caused some major foundational shift, but it was a solid contribution. Additionally, acquiring Mosaic fills a pretty gaping hole in their existing product offering. The whole process of pretraining, fine-tuning, and evaluating open source LLMs in production is a total nightmare currently. Mosaic's stack at least attempts to make that better (on top of releasing solid foundational models themselves). > “corporate leaders under pressure to get their data ready for AI”. That has nothing to do with LLMs I agree that its buzzwordy and a little abstract. But also in my experience, getting "data ready for AI" is actually the primary constraint many orgs have with respect to using LLMs in an enterprise context. Their data is not stored in a way thats easy to tokenize/label/embed for effective training or fine-tuning. And you could argue the preprocessing actually should the easy part for non-ML devs to tackle, as its primarily a software engineering problem. Yet, its still the thing that keeps many folks from getting started (before you even get to attempt to tackle the inference problem).
- ethbr0 3y ago
  From my understanding, Databrick's core value proposition is -- 'We let you do things you want to, with the people you have.' It may surprise HN to learn, but most companies don't have top-tier technical talent. Consequently, how do you do {cutting edge thing board and investors are demanding company do} with the people you have/can afford? Use an offering that decreases the necessary skill level by providing powerful prebuilt components. From this perspective, LLM integration sounds like a perfect fit. It's something every company is being asked to have a plan on, but one few are technically staffed to execute on their own.
  - stakhanov 3y ago
    > [...] LLM integration [...] is something every company is being asked to have a plan on, but one few are technically staffed to execute on their own. But this is still the "language of hype". LLM integration to do what? I'm not saying they're not doing anything useful, just that this is not a way to describe either a productivity-enhancing technology (feature) or an actual value proposition (benefit) in a meaningful way.
pletnes 3y ago
What has replaced Databricks, in your opinion? Doesn’t seem «dead» to me, yet. Serious question.
- aynyc 3y ago
  Their moat was to be able to spin up spark clusters for you. This was very useful when companies just starting in that space and don’t have the skills. But with K8s and clusters becoming more common for normal workload, it’s not hard to manage your own spark clusters anymore. There goes their moat. Their other offerings just aren’t that amazing. People now don’t just want to buy hand tools anymore, they want power tools. AI/ML and its cousins are in the power tool space. Databrick cost is stupid high, like snowflake. Any companies looking to chop budget would put those contracts up first.
  - disgruntledphd2 3y ago
    Their moat is their phenomenal sales team. Like, I'm at company 3 that pays for databricks. Amount of those companies that use Spark = 0. I've given up complaining now.
    - esafak 3y ago
      So what are they using, the notebooks?
      - pletnes 3y ago
        But in the notebooks you run pyspark / scala spark / spark sql code?
        esafak 3y ago
        Yes, you can, if you want.
        disgruntledphd2 3y ago
        Yes, but for most of these companies they don't actually run (much or any) spark stuff, they just use the notebooks. It's really depressing, tbh but I've made my peace with it now.
quickthrower2 3y ago
Looking at the Mosaic website, and looking at many AI startups, I can't help feeling like they are all small AWS Sagemaker, Bedrock, Trainium, ... (etc.) competitors, and not sure how they will compete with the sheer capital that Amazon has, and potential to offer that compute power cheaper (at least long enough to kill off most of the competition). Maybe it is it the off chance that one of these companies might become a big player worth billions that makes the odds worth it?
- george_123 3y ago
  A lot of AWS services (especially SageMaker) aren’t very good in customer experience. People buy them for nominal capabilities and AWS core bread and butter — short-term and long-term reliability. Most of these startups (AI and others) have to offer a compelling product before even being notable. Besides, AWS top level doesn’t care if you use sagemaker or not. There’s a premium but if you’re still using EC2 via another startup, they’re still capturing lions share of value.
  - dbish 3y ago
    Sagemaker is probably a bad example since most folks who run ML workloads on AWS don't use it (from what I've seen talking to many ML teams). It's partially a scattered focus on their customer type (are they for experts/non-experts/something in between?, what exact use cases are they covering?) and partially I think just bad UI/UX (related to customer focus). AWS will converge on what is working for ML platforms and just build their own as time goes on. First mover advantage won't matter and these earlier ML platforms will be consumed imho
- tim_sw 3y ago
  The devil is in the details. Training large LLMs requires a lot of custom infra (handling GPUs going down, efficiently pushing data to keep the accelators busy, deciding on which mechanism of parallelizing model training is better - data vs model parallelism or both, tuning hyperparams of optimizers which can be different for larger batch sizes, etc) Mosaic is one of the better providers for this. AWS is nowhere near ready at this current point in time, it is pretty much a "dumb" infra provider in large LLM training at this point. (Of course they won't be standing still and will prob acquire that capability one way or another)
fakedang 3y ago
Just curious, how is Databricks going to be irrelevant? I thought they were a reasonably powerful database provider for data analytics platforms in many companies?
- millenseed 3y ago
  Yeah that's an absolutely garbage idea. Databricks is basically the opposite of irrelevant at this point.

paulsutter 3y ago

This $1.3B dollar figure is surely relative to Databricks 2021 valuation from their Series H (most recent round), and doesn’t represent real money since multiples have plunged since then Estimates of Databricks revenue is $400M and “valuation” of $36B, which would never hold today

_bohm 3y ago
Where do you get the $400m figure? Just a couple weeks ago Bloomberg ran an article stating their revenue is >$1B
- noirbot 3y ago
  Yea, with what they charge, I'd be surprised if it was as low as 400 mil. I've seen mid-sized companies with 8-figure Databricks contracts.
tedivm 3y ago
Databricks wasn't the one purchased, MosaicML was. Mosaic's last round had a valuation of $136m, so this is a pretty big jump for them.
- gk1 3y ago
  The Databricks valuation is relevant if any part of the acquisition was paid for in Databricks equity.
  - jamesblonde 3y ago
    Any part? My guess is 100% of it.
brentm 3y ago
I heard Ali the CEO of Databricks speaking on a podcast recently and stated their revenue last year was north of 1B.

htrp 3y ago

Turns out Databricks couldn't do it all internally with just training the dolly model. I think it's a good accretive deal because Databricks already has a good number of enterprise relationships and Mosaics key offering has been training software with their own optimizations that increase gpu efficiency.

cheptsov 3y ago

Congrats to the MosaicML and wish all the luck! If anyone is interested in MosaicML alternatives, check out dstack, we are building its OSS and cloud-agnostic alternative: https://dstack.ai https://dstack.ai. Disclaimer: I'm the founder and CEO at dstack.

huac 3y ago
It would be really cool to support more vendors (eg Lambda Labs, Runpod, etc) - getting GPU availability feels like 'any port in a storm' sometimes :)
- cheptsov 3y ago
  Yup, we're already working on an experimental support for Lambda Labs! Ping us if you'd like to test it out. Perhaps we could show something already next week.

jgalt212 3y ago

> bringing down the cost of using generative AI—from tens of millions of dollars to hundreds of thousands of dollars per model, That's a lot of people you have to replace with one model--even if you get the cost basis down. Do those costs include the period refitting?

Tostino 3y ago
It really depends on what you are using it for. There are plenty of high-value use cases where you can free up resources that are spending time inefficiently due to handling tasks that a domain specific LLM could help with. Maybe that allows you to not expand your team by allowing your existing resources to work on things that bring more value.

poomer 3y ago

This class of startup, "build domain specific LLMS using your own data", is extremely crowded right now but I am not optimistic about their future. For large companies, the actual modeling work for this is already easy for any ML team, thanks to existing FOSS work on stuff like PEFT and LoRA. The hard part is figuring out what data goes into the fine tuning process and how to get this data in a usable form, but this is very business specific and can't be automated in a SaaS process. For SMBs, the value would be in using the LLM to generate responses to customer Q&A/search queries. But these companies aren't going to integrate some external third party service, they'll only use it if it's already baked into their CMS - Wordpress/Shopify/Wix/etc. I just don't see who the final consumer for this product would be.

re-thc 3y ago
> is extremely crowded right now but I am not optimistic about their future Why not? Every larger "cloud" company seems to be randomly buying 1 at the moment to offer "AI" so might get some good deals. This is clearly 1 of them - panic buy.
brucethemoose2 3y ago
> thanks to existing FOSS work on stuff like PEFT and LoRA YMMV. Sometimes a LORA is fine, but sometimes a full finetune is necessary for higher quality output. That being said, backwards pass free training keeps making more and more progress. Seems like a short matter of time before it becomes practical.
- Tostino 3y ago
  Look at QLoRA. The QLoRA can be attached to all layers, allowing you to alter behavior with much less data than the original LoRA implementation. It seems to "stick" better. I just fine tuned a ~30b parameter model on my 2x 3090s to check it out. It worked fantastically. I should be able to fine tune up-to 65b parameter models locally but wanted to get my dataset right on a smaller model before trying.
  - throwaway72762 3y ago

bratao 3y ago

Thrilled to hear about MosaicML's successful exit! Nonetheless, this could serve as an indication to explore other options. Given Databricks' past acquisition of Redash which led to its downfall, there's no absolute assurance that Mosaic's fate won't be similar.

orderedtree 3y ago
Why do you say it led to redash's downfall? What happened to redash since the acquisition?
- bratao 3y ago
  They've essentially killed Redash, which was one of the best open-source dashboard/data visualization tools. They had promised to keep it open-source and improve upon it (https://web.archive.org/web/20211019150919/https://blog.redash.io/redash-joins-databricks/ https://web.archive.org/web/20211019150919/https://blog.reda...). However, a year later, their SaaS has been discontinued. The open-source repository is now stagnant, with hundreds of unresolved merge requests. On a positive note, they recently shared the repository with some other open-source maintainers, so there's hope that Redash will be reborn.
  - justinclift 3y ago
    > On a positive note, they recently shared the repository with some other open-source maintainers, so there's hope that Redash will be reborn. Yeah, the root of the problem (in my opinion) was that Redash has sooo many Python dependencies (due to supporting so many databases and similar) that it's become a real hairball source code wise to keep them all playing together nicely. Especially as over time (say) library package FOO has some security vulnerability reported that gets fixed in a new release... but the dependencies of the newer release are too new to work with (say) package BAR. Times that by 50 and it's a real pita. Simultaneously to that, the Redash team got busy with their work at Databricks (mostly not Redash related). Then the automatic CircleCI checks on PRs started failing (ugh), etc. --- But, as @bratao mentioned above that's all getting worked through now. Admin and maintainer permissions have been given to a group of dev volunteers / known Redash enthusiasts. CI is working again now (as of last night), and we're currently untangling the dependency hairball. It's likely to be a few weeks (minimum I guess) before any new official releases are ready, but it will happen. :)

ShamelessC 3y ago

What do these companies do? I consider myself mildly well versed in the ML space at least, but I’ve never heard of MosaicML before (have they trained anything popular?). I’ve heard of Databricks but know very little about what service they provided before/after the recent deep learning craze.

Zizizizz 3y ago
Databricks provides Jupyter lab like notebooks for analysis and ETL pipelines using spark through pyspark, sparkql or scala. I think R is supported as well but it doesn't interop as well with their newer features as well as python and SQL do. It interfaces with cloud storage backend like S3 and offers some improvements to the parquet format of data querying that allows for updating, ordering and merged through https://delta.io https://delta.io . They integrate pretty seamlessly to other data visualisation tooling if you want to use it for that but their built in graphs are fine for most cases. They also have ML on rails type through menus and models if I recall but I typically don't use it for that. I've typically used it for ETL or ELT type workflows for data that's too big or isn't stored in a database.
- ShamelessC 3y ago
  Thanks!
- tsunamifury 3y ago
  Now say it even dumber as if I were a dog or a ceo.
  - quickthrower2 3y ago
    Perhaps, it is a fancy .csv importer?
  - esafak 3y ago
    They create services for data scientists (notebooks) and machine learning engineers (Spark).

underlines 3y ago

Coincidentally this comes after MosaicML released the best open source commercially usable LLMs on huggingface: mpt-30b, the first open source LLM with 8k context length that can be extended even further with ALiBi and has been trained on a whopping 1 trillion tokens vs. 300 billion for Pythia and OpenLLaMA, and 800 billion for StableLM.

jncraton 3y ago
OpenLLaMA models up to 13B parameters have now been trained on 1T tokens: https://github.com/openlm-research/open_llama https://github.com/openlm-research/open_llama
- underlines 3y ago
  unfortunately not openllama-33b yet
  - emadm 3y ago
    20b done
brucethemoose2 3y ago
Is it better than Falcon 40B?
swyx 3y ago
do we know how it ranks vs the other models yet? its not yet up on https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

brucethemoose2 3y ago

Falcon 7B/40B is good, but hasn't really caught on like LLaMA has. One big reason is that its not well supported by 4-bit inference code (namely llama.cpp and GPTQ). Mosaic's 7B (and 30B?) models have the same issue, and 7B kinda paled in comparison to LLaMA 7B... But maybe it would be better if finetuned? To me, its kinda baffling that Mosaic didn't work on adding highly quantized inference to the popular frameworks.

Tostino 3y ago
Falcon is also slow for inference compared to LLaMA models of similar size. Speed can turn into a quality all its own when you break up problems into small pieces and use the LLM to iterate.
- brucethemoose2 3y ago
  Thats because its unoptimized, right? My impression is that the Falcon GPTQ (and 8 bit bnb inference?) code is just immature.
ml_hardware 3y ago
Mosaic's MPT models are already supported in GGML: https://github.com/ggerganov/ggml https://github.com/ggerganov/ggml Here's MPT-30B running in 4-bit precision on CPU :) https://twitter.com/abacaj/status/1673133443339763712?s=20 https://twitter.com/abacaj/status/1673133443339763712?s=20
- brucethemoose2 3y ago
  Oh I didn't realize this. Everything is moving so fast that I can't even keep up with the features.

sqs 3y ago

Congratulations to folks at MosaicML and Databricks!

YChacker100 3y ago

Congratulations to all involved!!

woeirua 3y ago

How old was MosaicML? How many customers did they have? Revenue? Profits? This feels like pure hype for LLMs to be honest. Good for the founders of MosaicML to exit at the top, but I'd bet good money that DataBricks winds up writing this off in a year or two max.

swyx 3y ago

huge congrats to the team - we interviewed Jonathan and Abhi from the team last month ( https://www.latent.space/p/mosaic-mpt-7b#details https://www.latent.space/p/mosaic-mpt-7b#details ) and it was blindingly obvious that they were an incredible team of engineers building one of the most valuable training platforms in the industry. congrats!

CrimsonWanderer 3y ago
Good to hear they are responsive to PR opportunities. Too bad their interview process doesn't match this level of enthusiasm. I had a recruiter reach out from DB late last year. I wasn't looking for new work but it seemed like it'd be worth a chat. I had to move my schedule around to fit it in and then get up extra early to be prepared for the chat...only the recruiter never showed. Didn't follow up about missing a meeting they had scheduled. It's been crickets. That is enough of a lack of professionalism really stood out as a red flag. Hard pass.

jedberg 3y ago

I didn't realize Databricks was doing so well they could spare $1B in equity/cash. Congrats not only to MosaicML but Databricks as well!

hackitup7 3y ago
Yeah, they're one of the strongest private software companies right now. I'd say easily top 10.
- dbish 3y ago
  It's so suprising to me that there's enough dropped on the floor/use cases ignored from AWS/Azure that they could build such a large business.

cavisne 3y ago

Wonder what the outcome for early employees looks like here? This is not a very big company and hasn't been around for very long. 1.3B is a lot of money

ttymck 3y ago

~50 employees? I'd love to hear what regular employees will see from this (if anything). Even if it's completely controlled by NDA, I think even confirming the existence of that NDA would be helpful. So often we see these "success" stories and we don't hear exactly how the "regular joes" made out. And it can color our perception of the startup gamble, without real evidence.

DropInIn 3y ago

In light of the story I see just above this one ( https://news.ycombinator.com/item?id=36475376 https://news.ycombinator.com/item?id=36475376 ) Imma call inflated bs....

OthmaneHamzaoui 3y ago

I'm less surprised by the acquisition but more confused by seeing MosaicML labeled as 'Generative AI startup'. To my knowledge MosaicML is an open source lib for optimising training/inference taking a "speedup" tricks from research papers and making them available easily. At least a few months back that was the case. When did they become a Gen AI startup ? Or they rebranded lately?

zht 3y ago

Unless if this 1.3B cash I wouldn’t celebrate so soon. This is likely mostly stock deal, backed by databricks equity that, like most other private companies, is illiquid and has taken a nose dive in value in the last couple of years. I don’t know the exact split, but I’d imagine it is 300m cash, 1B in databricks stock at some last round private valuation that’s likely untested Databricks last raises at 38B valuation at 1B ARR near peak valuation hype in August 2021. Assuming they can get 10x and their ARR has doubled since then, they’re down maybe 50 percent in valuation. Internally they say they are marking themselves down to about 34B. Is the deal using databricks shares valued at 38B? 34B? 20B? Who knows. I hope it’s the latter. And I hope that whenever Mosaic folks get to sell their databricks stocks that it is worth more than that

j_shi 3y ago

Actually seems Databricks got a great deal for Mosaic. Real qustion is why Mosaic took it v. hold out or do another round Rough math plugging in public #s and comments here: - All stock deal at Aug 2021 val of 38B (1B ARR) - Assume rev doubled to 2B (which may even be aggressive) - SAAS multiples are down 6x since Aug 2021 - 38B x 2 / 6 = $12.7B - 12.7B / 38B * 1.3B = 434M = effective price - Assume 100M to pref stock --> Comes out to 334M, with a chunk of that (1/3? 1/4?) potentially subject to earn out

j_shi 3y ago
Realized there's probably pref on Databricks too, which would further lower the value of its common. On the other hand, there could have been a markdown from the 38B since August '21