4 ms·

People are excited about Snowflake because it can completely disrupt the traditional data-warehouse market. The legacy players like Teradata and Exadata (from

People are excited about Snowflake because it can completely disrupt the traditional data-warehouse market. The legacy players like Teradata and Exadata (from Oracle) really don't scale. Teradata has ~2B in revenue, Exadata is probably in the same range. That's all up for grabs but that's only scratching the surface. Historically, only transactional data was dumped into the warehouse. Snowflake is selling storage at S3 price (plus you get compression so often ends up cheaper) while they are making money of compute/query. If they can provide all the right query abstractions (SQL, full-text search), in theory all data can be thrown in Snowflake. Yes, tech savy bay area companies can setup their own stack using Presto etc but rest of the world is not like that.

ETHisso2017 6y ago
>>>Yes, tech savy bay area companies can setup their own stack using [insert open source tool here] etc but rest of the world is not like that. It sounds like a ton of these cloud infra companies have this product strategy (datadog, snowflake, elastic, hashicorp, etc)
- Spartan-S63 6y ago
  When you look at cloud infra companies like that, their competitive advantage is in quickly being able to ingest data and make it accessible, so an off-the-shelf solution likely doesn't exist for their particular use-case. Also, since that operation is your competitive advantage, you should look to in-source it rather than reach for a COTS solution.
hilbertseries 6y ago
> Yes, tech savy bay area companies can setup their own stack using Presto etc but rest of the world is not like that. My last company was an early adopter of Snowflake. And we tried Presto first, circa 2016 and Presto was sloooow. We were using vertica at the time and it was so much slower. Snowflake on the other hand was able to perform on the same order of latency as Vertica, which was pretty crazy to us.
- kwillets 6y ago
  So why did you switch from Vertica (or did you)?
  - hilbertseries 6y ago
    Vertica was too expensive, their licensing fees were terrible at our scale. Operations were also awful, if we had two nodes go down we were always in trouble. We built an EBS solution that made it a little better, but it still wasn’t tenable long term.
    - kwillets 6y ago
      Good info -- thanks. Their Eon mode product is very similar to Snowflake, with S3 storage and semi-dynamic compute nodes, but they may not be as slick at marketing it or providing a UI.
      - detroitcoder 6y ago
        Yes Eon is similar BUT it is not nearly as turnkey. Agree it is not well marketed.
  - bydlocoder 6y ago
    My previous company still uses Vertica (on-prem) but it still wasn't as fast as it should be. Trying to hire anyone for an operations team was an enterprise in futility, since it's a rather niche tech. Maybe it would've been better in the cloud, but here large companies are cautious about vendor lock-in and after 2014, potential impact of sanctions.
- soumyadeb 6y ago
  That's interesting. I thought Vertica's pitch was real-time analytics for which draditional disk based data-warehouses are too slow.
  - hilbertseries 6y ago
baskire 6y ago
Smaller companies with presto won’t get the same performance benefit. Snowflake & BigQuery get the ability to have multiple customers on a large cluster. It’d be cost prohibitive for a single smaller customer to have all that compute sitting idle for a few queries per minute. Storage also benefits as snowflake/ BQ can shard your data across a much larger array of disk giving you better IO. Think is it faster to drive a car 100 ft starting at 0mph and flooring it. Or to drive 100ft with a car which starts off doing 120mph
CharlesW 6y ago
[Teradata employee here.] > The legacy players like Teradata and Exadata (from Oracle) really don't scale. I get why Teradata gets labelled "legacy", but one of Teradata's main differentiators is scale. Teradata engineers have been tackling incredibly interesting scale problems (on many dimensions of "scale") for 40 years. Teradata has many customers who routinely manage and perform analytics on many petabytes of data. > Historically, only transactional data was dumped into the warehouse. That was once true, because initially that was all the data that companies had. However, companies have long since used data warehouses for all kinds of data — sensor data, text, behavioral data, product info/BOMs, vendor info, contract info, etc. — whatever's necessary to run the business. > Snowflake is selling storage at S3 price… This is important, but not unique. For example, Teradata's current product has native support for S3 and S3-compatible object stores, and you can query them just like any other database table, join that data with data in high-performance native storage, etc.
- soumyadeb 6y ago
  Sorry, I didn't clarify well. I am sure it scales technically well but not on cost. My experience of TD is > 10 yrs and then the multi-node version was substantially more expensive than the single-node version. Also, storage and compute was coupled which meant I had to pay for nodes even if 99% of my data was cold. That's a problem with RedShift too but not for Snowflake. De-coupling storage and compute was a brilliant move by Snowflake. BigQuery can completely abstracted compute - you don't provision compute and only pay for data scanned. However, it gives you a sense of insecurity around cost - A single bad cron job running a query every sec can blow up your cost (real-life experience). Snowflake provides the best cost/performance tradeoff I have seen.
  - CharlesW 6y ago
    > I am sure it scales technically well but not on cost. The honest answer is "it depends". Because Teradata is a different beast, per-query pricing can be significantly cheaper than Snowflake with high-volume workloads. It's worth trying both to evaluate cost and performance. > Also, storage and compute was coupled which meant I had to pay for nodes even if 99% of my data was cold. Yes, it used to be that everything had to into Teradata's high-performance filesystem. These days, Vantage's native object storage support means that you can keep that cold data in S3.
bawolff 6y ago
Ironic that products named exadata and teradata (guess we skipped petadata) dont scale.

ETHisso2017 6y ago

>>>Yes, tech savy bay area companies can setup their own stack using [insert open source tool here] etc but rest of the world is not like that. It sounds like a ton of these cloud infra companies have this product strategy (datadog, snowflake, elastic, hashicorp, etc)

Spartan-S63 6y ago
When you look at cloud infra companies like that, their competitive advantage is in quickly being able to ingest data and make it accessible, so an off-the-shelf solution likely doesn't exist for their particular use-case. Also, since that operation is your competitive advantage, you should look to in-source it rather than reach for a COTS solution.

hilbertseries 6y ago

> Yes, tech savy bay area companies can setup their own stack using Presto etc but rest of the world is not like that. My last company was an early adopter of Snowflake. And we tried Presto first, circa 2016 and Presto was sloooow. We were using vertica at the time and it was so much slower. Snowflake on the other hand was able to perform on the same order of latency as Vertica, which was pretty crazy to us.

kwillets 6y ago
So why did you switch from Vertica (or did you)?
- hilbertseries 6y ago
  Vertica was too expensive, their licensing fees were terrible at our scale. Operations were also awful, if we had two nodes go down we were always in trouble. We built an EBS solution that made it a little better, but it still wasn’t tenable long term.
  - kwillets 6y ago
    Good info -- thanks. Their Eon mode product is very similar to Snowflake, with S3 storage and semi-dynamic compute nodes, but they may not be as slick at marketing it or providing a UI.
    - detroitcoder 6y ago
      Yes Eon is similar BUT it is not nearly as turnkey. Agree it is not well marketed.
- bydlocoder 6y ago
  My previous company still uses Vertica (on-prem) but it still wasn't as fast as it should be. Trying to hire anyone for an operations team was an enterprise in futility, since it's a rather niche tech. Maybe it would've been better in the cloud, but here large companies are cautious about vendor lock-in and after 2014, potential impact of sanctions.
soumyadeb 6y ago
That's interesting. I thought Vertica's pitch was real-time analytics for which draditional disk based data-warehouses are too slow.
- hilbertseries 6y ago

baskire 6y ago

Smaller companies with presto won’t get the same performance benefit. Snowflake & BigQuery get the ability to have multiple customers on a large cluster. It’d be cost prohibitive for a single smaller customer to have all that compute sitting idle for a few queries per minute. Storage also benefits as snowflake/ BQ can shard your data across a much larger array of disk giving you better IO. Think is it faster to drive a car 100 ft starting at 0mph and flooring it. Or to drive 100ft with a car which starts off doing 120mph

CharlesW 6y ago

[Teradata employee here.] > The legacy players like Teradata and Exadata (from Oracle) really don't scale. I get why Teradata gets labelled "legacy", but one of Teradata's main differentiators is scale. Teradata engineers have been tackling incredibly interesting scale problems (on many dimensions of "scale") for 40 years. Teradata has many customers who routinely manage and perform analytics on many petabytes of data. > Historically, only transactional data was dumped into the warehouse. That was once true, because initially that was all the data that companies had. However, companies have long since used data warehouses for all kinds of data — sensor data, text, behavioral data, product info/BOMs, vendor info, contract info, etc. — whatever's necessary to run the business. > Snowflake is selling storage at S3 price… This is important, but not unique. For example, Teradata's current product has native support for S3 and S3-compatible object stores, and you can query them just like any other database table, join that data with data in high-performance native storage, etc.

soumyadeb 6y ago
Sorry, I didn't clarify well. I am sure it scales technically well but not on cost. My experience of TD is > 10 yrs and then the multi-node version was substantially more expensive than the single-node version. Also, storage and compute was coupled which meant I had to pay for nodes even if 99% of my data was cold. That's a problem with RedShift too but not for Snowflake. De-coupling storage and compute was a brilliant move by Snowflake. BigQuery can completely abstracted compute - you don't provision compute and only pay for data scanned. However, it gives you a sense of insecurity around cost - A single bad cron job running a query every sec can blow up your cost (real-life experience). Snowflake provides the best cost/performance tradeoff I have seen.
- CharlesW 6y ago
  > I am sure it scales technically well but not on cost. The honest answer is "it depends". Because Teradata is a different beast, per-query pricing can be significantly cheaper than Snowflake with high-volume workloads. It's worth trying both to evaluate cost and performance. > Also, storage and compute was coupled which meant I had to pay for nodes even if 99% of my data was cold. Yes, it used to be that everything had to into Teradata's high-performance filesystem. These days, Vantage's native object storage support means that you can keep that cold data in S3.

bawolff 6y ago

Ironic that products named exadata and teradata (guess we skipped petadata) dont scale.