5 ms·

...a talk with your local Dev Ops Engineer/ manager to discuss how to secure your implementations This right here is exactly why Snowflake is a good fit for my

...a talk with your local Dev Ops Engineer/ manager to discuss how to secure your implementations This right here is exactly why Snowflake is a good fit for my org.: we could pay someone's salary to install, maintain, and upgrade some open source alternative (and the VPS to run it), or we could just pay for Snowflake and stop wasting engineering's time on the Data Science's team's stuff, which frees them up to move faster on the core platform features bringing in the big bucks. This reads like a flavor of the same argument I've heard many times over. You know what the biggest cost of an org. really is as we head into a recession? Labor. Stop wasting it on stuff done better by specialized companies that free you up to get stuff done.

rglullis 3y ago
> we could pay someone's salary to install, maintain, and upgrade some open source alternative (and the VPS to run it), or we could just pay for Snowflake and stop wasting engineering's time You are missing the "we could pay someone that provides a commodified, open-source alternative to Snowflake" option. It would be cheaper, it wouldn't require any of your engineering team resources and it would help us all get rid of single vendor lock-in.
- bushbaba 3y ago
  Would it be cheaper? Cloudera fits the bill and it’s for sure not cheaper
  - rglullis 3y ago
    Because Cloudera is aimed at a market that is not the same that would benefit from "most companies" being talked about in TFA? Plenty of companies that would be fine by using a managed postgresql database for their "data warehouse" and you can get, e.g, one reasonably powerful server at Digital Ocean for $60/month with managed backups - and that already has quite a big markup compared to their droplets because they are just packaging some monitoring tools on top of their existing systems.
- hodgesrm 3y ago
  > You are missing the "we could pay someone that provides a commodified, open-source alternative to Snowflake" option. Which services provider do you have in mind to pay? The advantage of Snowflake is that the CIO can pick Snowflake as the default for the entire org and it will work across a very large number of business use cases. I don't see any open source analytic software that can do that. In my experience open source analytic solutions work much better for specific use cases, say network flow log analysis. You pick exactly the right set of components for that use case and build specifically to the problem. Maybe you use ClickHouse + Kafka--cheaper, simpler, faster. But when you shift to another problem, such as analyzing support case patterns, that requires unpredictable joins on large numbers of tables the first solution does not work at all. Now you need something like Presto on data lakes, which is completely different technology. The alternative is to move all the data in to Snowflake and do both of the above use cases there. Sure, it's not the fastest or the cheapest software but the savings in labor more than make up for the licensing cost. A lot of companies make that decision.
  - rglullis 3y ago
    > The advantage of Snowflake is that the CIO can pick Snowflake as the default for the entire org We are talking about companies that do not even have a CIO, which is just another way to say "most companies".
- notatoad 3y ago
  >it wouldn't require any of your engineering team resources other than the resources to evaluate the commodified open-source alternative. and the resources to manage the risk of that alternative, and the smaller company who runs it, going away. "nobody ever got fired for buying IBM" is usually said in a snarky way, but there's real value in not only not having to think about the provider, but not having to think about which provider to choose. making that decision has a cost, and the more off the beaten path you go the higher the cost of that decision.
  - rglullis 3y ago
    > "nobody ever got fired for buying IBM" https://en.wikipedia.org/wiki/File:Survivorship-bias.svg https://en.wikipedia.org/wiki/File:Survivorship-bias.svg How many startups simply died or how many projects never got to be launched because they thought they needed some enterprise support from the start?
ramesh31 3y ago
>This right here is exactly why Snowflake is a good fit for my org.: we could pay someone's salary to install, maintain, and upgrade some open source alternative (and the VPS to run it), or we could just pay for Snowflake and stop wasting engineering's time on the Data Science's team's stuff, which frees them up to move faster on the core platform features bringing in the big bucks. The cost problem really is out of control. You'd come out ahead financially pretty quickly with a DIY solution. DBUs are on the order of ~100x the actual underlying compute cost. I simply cannot understand how anyone chooses this over running your own Spark clusters with Jupyterlab. There are single-click image deploys that do this stuff.
- ironchef 3y ago
  Here was my situation. Occasional queries. Over a couple petabyte of data. Customer facing so response in seconds per SLA but > 95 percent of the time the warehouse isn’t running. Cached queries from within 24 hours which don’t require the warehouse to even spin up. Our snowflake costs were significantly less than an FTE. Would that potentially be a situation which “running your own” doesn’t make sense?
  - agent281 3y ago
    If you have petabytes of data, I don't think this article is talking about your use case.
    - sanderjd 3y ago
      I think it is? Or I guess, what data size do you think it's talking about? If you only have gigabytes of data, none of this matters, you can use anything pretty cheaply and easily. So is this article just for "terabytes" or does it go up to "hundreds of terabytes" but not "petabytes"?
      - agent281 3y ago
        Hmm, I suppose it's a bit challenging to say. I initially thought that it wasn't for the 80% smallest companies and petabytes of data is probably puts you in the top 20%. (Most businesses are small businesses after all.) However, I now realize that th biggest companies probably should manage their own data. If you're Google why would you use Snowflake? So I don't know if you are the target audience for this blog post. It's pretty ambiguous.

rglullis 3y ago

> we could pay someone's salary to install, maintain, and upgrade some open source alternative (and the VPS to run it), or we could just pay for Snowflake and stop wasting engineering's time You are missing the "we could pay someone that provides a commodified, open-source alternative to Snowflake" option. It would be cheaper, it wouldn't require any of your engineering team resources and it would help us all get rid of single vendor lock-in.

bushbaba 3y ago
Would it be cheaper? Cloudera fits the bill and it’s for sure not cheaper
- rglullis 3y ago
  Because Cloudera is aimed at a market that is not the same that would benefit from "most companies" being talked about in TFA? Plenty of companies that would be fine by using a managed postgresql database for their "data warehouse" and you can get, e.g, one reasonably powerful server at Digital Ocean for $60/month with managed backups - and that already has quite a big markup compared to their droplets because they are just packaging some monitoring tools on top of their existing systems.
hodgesrm 3y ago
> You are missing the "we could pay someone that provides a commodified, open-source alternative to Snowflake" option. Which services provider do you have in mind to pay? The advantage of Snowflake is that the CIO can pick Snowflake as the default for the entire org and it will work across a very large number of business use cases. I don't see any open source analytic software that can do that. In my experience open source analytic solutions work much better for specific use cases, say network flow log analysis. You pick exactly the right set of components for that use case and build specifically to the problem. Maybe you use ClickHouse + Kafka--cheaper, simpler, faster. But when you shift to another problem, such as analyzing support case patterns, that requires unpredictable joins on large numbers of tables the first solution does not work at all. Now you need something like Presto on data lakes, which is completely different technology. The alternative is to move all the data in to Snowflake and do both of the above use cases there. Sure, it's not the fastest or the cheapest software but the savings in labor more than make up for the licensing cost. A lot of companies make that decision.
- rglullis 3y ago
  > The advantage of Snowflake is that the CIO can pick Snowflake as the default for the entire org We are talking about companies that do not even have a CIO, which is just another way to say "most companies".
notatoad 3y ago
>it wouldn't require any of your engineering team resources other than the resources to evaluate the commodified open-source alternative. and the resources to manage the risk of that alternative, and the smaller company who runs it, going away. "nobody ever got fired for buying IBM" is usually said in a snarky way, but there's real value in not only not having to think about the provider, but not having to think about which provider to choose. making that decision has a cost, and the more off the beaten path you go the higher the cost of that decision.
- rglullis 3y ago
  > "nobody ever got fired for buying IBM" https://en.wikipedia.org/wiki/File:Survivorship-bias.svg https://en.wikipedia.org/wiki/File:Survivorship-bias.svg How many startups simply died or how many projects never got to be launched because they thought they needed some enterprise support from the start?

ramesh31 3y ago

>This right here is exactly why Snowflake is a good fit for my org.: we could pay someone's salary to install, maintain, and upgrade some open source alternative (and the VPS to run it), or we could just pay for Snowflake and stop wasting engineering's time on the Data Science's team's stuff, which frees them up to move faster on the core platform features bringing in the big bucks. The cost problem really is out of control. You'd come out ahead financially pretty quickly with a DIY solution. DBUs are on the order of ~100x the actual underlying compute cost. I simply cannot understand how anyone chooses this over running your own Spark clusters with Jupyterlab. There are single-click image deploys that do this stuff.

ironchef 3y ago
Here was my situation. Occasional queries. Over a couple petabyte of data. Customer facing so response in seconds per SLA but > 95 percent of the time the warehouse isn’t running. Cached queries from within 24 hours which don’t require the warehouse to even spin up. Our snowflake costs were significantly less than an FTE. Would that potentially be a situation which “running your own” doesn’t make sense?
- agent281 3y ago
  If you have petabytes of data, I don't think this article is talking about your use case.
  - sanderjd 3y ago
    I think it is? Or I guess, what data size do you think it's talking about? If you only have gigabytes of data, none of this matters, you can use anything pretty cheaply and easily. So is this article just for "terabytes" or does it go up to "hundreds of terabytes" but not "petabytes"?
    - agent281 3y ago
      Hmm, I suppose it's a bit challenging to say. I initially thought that it wasn't for the 80% smallest companies and petabytes of data is probably puts you in the top 20%. (Most businesses are small businesses after all.) However, I now realize that th biggest companies probably should manage their own data. If you're Google why would you use Snowflake? So I don't know if you are the target audience for this blog post. It's pretty ambiguous.