6 ms·

Snowflake seems like it's been losing relevance ever since Clickhouse became more popular. Seems like they're struggling to maintain performance vs the other c

Snowflake seems like it's been losing relevance ever since Clickhouse became more popular. Seems like they're struggling to maintain performance vs the other competitors out there. Not sure how this acquisition will help here. I don't think decoupling storage and compute was a good bet in the long run.

CharlesW 3y ago
Microsoft is also taking shots across Snowflake's bow with solutions like Microsoft Fabric.
quadrature 3y ago
Thats interesting, i don't see these as occupying the same space. Clickhouse is in the space of realtime analytics and Snowflake is a data warehouse. Although you could use Clickhouse for similar things it will fail at doing large distributed joins and similarly Snowflake will have trouble meeting a subsecond SLO. also FWIW Clickhouse's cloud offering also decouples storage and compute using an object store, but they found a good middleground where they keep local caches of hot data.
- riku_iki 3y ago
  > doing large distributed joins but ch supports large distributed joins?..
  - quadrature 3y ago
    Thanks, looks like i need to update my priors.
    - atwong 3y ago
      not really. CH times out on TPCH join scenarios. https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detailed-comparison https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detail...
      - riku_iki 3y ago
        that link doesn't provide much details how they try to test ch for joins, and if they tried to test it at all..
        qoega 3y ago
        I think atwong just promotes his product https://news.ycombinator.com/threads?id=atwong https://news.ycombinator.com/threads?id=atwong
- berkle4455 3y ago
  ch cluster works just fine on large distributed joins
  - atwong 3y ago
    not really. CH clusters timeout on join testing on TPCH. https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detailed-comparison https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detail...
- FridgeSeal 3y ago
  But CH is capable of the same “data warehousing” features that snowflake is. Which leaves snowflake as a slower, less capable, less open, and more expensive alternative. Which brings me to the next point: I’m convinced the delineation between “data warehouse” and “olap” is largely a marketing move designed to segment the market along made up boundaries.
  - sv123 3y ago
    We use both MS SQL and Snowflake heavily. There are clearly instances where having row based storage is appropriate, and also instances where columnar storage outperforms. All based on your workload and not just marketing.
    - berkle4455 3y ago
      I don't think anyone disagrees with you. Clickhouse and Snowflake are both OLAP; neither are row-based (OLTP).
    - FridgeSeal 3y ago
      MSSQL is an OLTP based db (going to preclude discussion of its fancy column index stuff it’s capable of). OLTP db’s definitely, definitely have a different role. I’m talking about the false difference between the likes of ClickHouse and Snowflake, where they’re both column oriented already. I’m asserting that the fundamental differences between “classic” column db’s and “data warehouses” is far less fundamental than the marketing would have us believe. Some of the db’s in this space have slightly different architectures and trade offs, and some deliberately operate at different scales, but they are built for, and operate in, basically the same purpose.
    - fuy 3y ago
      There's columnar tables in SQL Server, though. Have you tried it? Would be interesting to compare to Snowflake.
      - FridgeSeal 3y ago
        As useful and powerful as the columnar tables in mssql are, they’re not on the same level as a full columnar db.
fhoffa 3y ago
"More popular"? Citation needed, please. In terms of measuring popularity, I love https://db-engines.com/en/ranking https://db-engines.com/en/ranking Google Trends is interesting too https://trends.google.com/trends/explore?date=2021-04-24%202023-05-24&geo=US&q=%2Fg%2F11b8krtt2g,clickhouse&hl=en https://trends.google.com/trends/explore?date=2021-04-24%202... Disclosure: I work for Snowflake
- vgt 3y ago
  don't disagree with what you said, but your Google Trends argument has a big asterisk against it - right in the page it says "This comparison contains both Search terms and Topics, which are measured differently. LEARN MORE"
  - fhoffa 3y ago
    I hear what you say. But does it change anything?
    - vgt 3y ago
      Yes, efficacy of your argument
      - chimerasaurus 3y ago
        As a data point, if you examine something more granular and trend/topic tied, like Snowpark (which is close to Clickhouse alone) or "Snowflake Table" I would propose the overall point being made kind of stands. The original term is ambiguous (I wish Snowflake had different branding) but more specific terms to Snowflake still rank high and are maybe less wonky of a comparison.
- booi 3y ago
  Oracle is the most popular DB? I’ve never run into an Oracle DB in my entire career
  - 3y ago
skeeter2020 3y ago
These are pretty different products with different use cases IME. I haven't used clickhouse in production but we use Snowflake extensively and I'm a big fan of the product and the business model. The ecosystem also seems to be in sync with the needs of people building on top of Snowflake as well.
ignoramous 3y ago
It isn't just clickhouse, but databricks (delta.io) breathing their necks too, who incidentally have their own open LLM viz Dolly
flappyeagle 3y ago
SF and CH don’t seem like competitive products to me. What am I missing?
- berkle4455 3y ago
  They're both data warehouses that do a great job operating on massive datasets and neither should be your primary source of truth. I guess my question to you is: Why aren't they competitive products?
  - coding123 3y ago
    CH doesn't have this, for one: https://app.snowflake.com/marketplace/ https://app.snowflake.com/marketplace/
    - atwong 3y ago
      You could argue that it's an addon. OLAP databases just allow you to do SQL for analytical use cases.
btown 3y ago
What's more expensive: the data engineering staff you need to have on hand to optimize data loading and queries all to make sure your Snowflake/Databricks bill doesn't balloon out of control, or the staff to maintain your data on either cloud or self-hosted Clickhouse for equal or better query performance? In a world of limitless VC money, one might choose the more familiar and battle-tested Snowflake dynamics every time... but the world is shifting quite rapidly, and the degree to which investment in a Clickhouse stack is much less likely to "trap" you in rapidly expanding spend on a more closed ecosystem is becoming notable.
te_chris 3y ago
What I don’t get is why people use SF when BQ exists.
- ramraj07 3y ago
  Cheaper (yes), computational and billing model that's sufficiently different to mean something, works on AWS.

CharlesW 3y ago

Microsoft is also taking shots across Snowflake's bow with solutions like Microsoft Fabric.

quadrature 3y ago

Thats interesting, i don't see these as occupying the same space. Clickhouse is in the space of realtime analytics and Snowflake is a data warehouse. Although you could use Clickhouse for similar things it will fail at doing large distributed joins and similarly Snowflake will have trouble meeting a subsecond SLO. also FWIW Clickhouse's cloud offering also decouples storage and compute using an object store, but they found a good middleground where they keep local caches of hot data.

riku_iki 3y ago
> doing large distributed joins but ch supports large distributed joins?..
- quadrature 3y ago
  Thanks, looks like i need to update my priors.
  - atwong 3y ago
    not really. CH times out on TPCH join scenarios. https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detailed-comparison https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detail...
    - riku_iki 3y ago
      that link doesn't provide much details how they try to test ch for joins, and if they tried to test it at all..
      - qoega 3y ago
        I think atwong just promotes his product https://news.ycombinator.com/threads?id=atwong https://news.ycombinator.com/threads?id=atwong
berkle4455 3y ago
ch cluster works just fine on large distributed joins
- atwong 3y ago
  not really. CH clusters timeout on join testing on TPCH. https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detailed-comparison https://celerdata.com/blog/clickhouse-vs.-starrocks-a-detail...
FridgeSeal 3y ago
But CH is capable of the same “data warehousing” features that snowflake is. Which leaves snowflake as a slower, less capable, less open, and more expensive alternative. Which brings me to the next point: I’m convinced the delineation between “data warehouse” and “olap” is largely a marketing move designed to segment the market along made up boundaries.
- sv123 3y ago
  We use both MS SQL and Snowflake heavily. There are clearly instances where having row based storage is appropriate, and also instances where columnar storage outperforms. All based on your workload and not just marketing.
  - berkle4455 3y ago
    I don't think anyone disagrees with you. Clickhouse and Snowflake are both OLAP; neither are row-based (OLTP).
  - FridgeSeal 3y ago
    MSSQL is an OLTP based db (going to preclude discussion of its fancy column index stuff it’s capable of). OLTP db’s definitely, definitely have a different role. I’m talking about the false difference between the likes of ClickHouse and Snowflake, where they’re both column oriented already. I’m asserting that the fundamental differences between “classic” column db’s and “data warehouses” is far less fundamental than the marketing would have us believe. Some of the db’s in this space have slightly different architectures and trade offs, and some deliberately operate at different scales, but they are built for, and operate in, basically the same purpose.
  - fuy 3y ago
    There's columnar tables in SQL Server, though. Have you tried it? Would be interesting to compare to Snowflake.
    - FridgeSeal 3y ago
      As useful and powerful as the columnar tables in mssql are, they’re not on the same level as a full columnar db.

fhoffa 3y ago

"More popular"? Citation needed, please. In terms of measuring popularity, I love https://db-engines.com/en/ranking https://db-engines.com/en/ranking Google Trends is interesting too https://trends.google.com/trends/explore?date=2021-04-24%202023-05-24&geo=US&q=%2Fg%2F11b8krtt2g,clickhouse&hl=en https://trends.google.com/trends/explore?date=2021-04-24%202... Disclosure: I work for Snowflake

vgt 3y ago
don't disagree with what you said, but your Google Trends argument has a big asterisk against it - right in the page it says "This comparison contains both Search terms and Topics, which are measured differently. LEARN MORE"
- fhoffa 3y ago
  I hear what you say. But does it change anything?
  - vgt 3y ago
    Yes, efficacy of your argument
    - chimerasaurus 3y ago
      As a data point, if you examine something more granular and trend/topic tied, like Snowpark (which is close to Clickhouse alone) or "Snowflake Table" I would propose the overall point being made kind of stands. The original term is ambiguous (I wish Snowflake had different branding) but more specific terms to Snowflake still rank high and are maybe less wonky of a comparison.
booi 3y ago
Oracle is the most popular DB? I’ve never run into an Oracle DB in my entire career
- 3y ago

skeeter2020 3y ago

These are pretty different products with different use cases IME. I haven't used clickhouse in production but we use Snowflake extensively and I'm a big fan of the product and the business model. The ecosystem also seems to be in sync with the needs of people building on top of Snowflake as well.

ignoramous 3y ago

It isn't just clickhouse, but databricks (delta.io) breathing their necks too, who incidentally have their own open LLM viz Dolly

flappyeagle 3y ago

SF and CH don’t seem like competitive products to me. What am I missing?

berkle4455 3y ago
They're both data warehouses that do a great job operating on massive datasets and neither should be your primary source of truth. I guess my question to you is: Why aren't they competitive products?
- coding123 3y ago
  CH doesn't have this, for one: https://app.snowflake.com/marketplace/ https://app.snowflake.com/marketplace/
  - atwong 3y ago
    You could argue that it's an addon. OLAP databases just allow you to do SQL for analytical use cases.

btown 3y ago

What's more expensive: the data engineering staff you need to have on hand to optimize data loading and queries all to make sure your Snowflake/Databricks bill doesn't balloon out of control, or the staff to maintain your data on either cloud or self-hosted Clickhouse for equal or better query performance? In a world of limitless VC money, one might choose the more familiar and battle-tested Snowflake dynamics every time... but the world is shifting quite rapidly, and the degree to which investment in a Clickhouse stack is much less likely to "trap" you in rapidly expanding spend on a more closed ecosystem is becoming notable.

te_chris 3y ago

What I don’t get is why people use SF when BQ exists.

ramraj07 3y ago
Cheaper (yes), computational and billing model that's sufficiently different to mean something, works on AWS.