4 ms·
Amazon Redshift has some architectural problems in an enterprise environment that Snowflake solves. Redshift does not scale up well - everything has to fit on o
by sol_remmy 9y ago
Amazon Redshift has some architectural problems in an enterprise environment that Snowflake solves. Redshift does not scale up well - everything has to fit on one cluster and eventually you hit limits.
When a Redshift cluster gets too big:
1. Takes forever to take backups, create read-replicas, modify anything
2. Redshift clusters can still only handle X concurrent queries, no matter how big the cluster is
3. Also Redshift is dragging their feet on adding new features i.e. Active Directory support.
Snowflake clusters allow a theoretically unlimited amount of users to query the same tables in a database - so it scales up forever.
And each team can create their own "compute cluster" which allows teams to track and divide costs much more easily then trying to associate this-batch-user on Redshift with this-team/line-of-business.
Snowflake is well-designed for an enterprise environment where line-of-businesses need to share/access each others data but still have separate infrastructure. Redshift does not make it easy go between Redshift clusters.
Also Snowflake as a company is still hungry and is willing to add features if you pay enough
EDIT: However Snowflake is more expensive than Redshift
EDIT: Splitting data between two Redshift clusters was not efficient for us and trying to have the same table in multiple Redshift clusters requires you creating your own tooling and the data might not stay in sync so the only real option was keeping everything in one massive cluster
- georgewfraser 9y agoWhat makes you think snowflake is more expensive? When I have tested it, it has given higher performance / $. Have you seen different results with your workload?
- sol_remmy 9y agoIf I remember correctly, Redshift became more cost-effective over Snowflake the larger your compute got. Snowflake would just run an EC2 instance in the background for compute, and the very largest EC2 instances were more expensive to run than a Redshift cluster of similar power.
- georgewfraser 9y agoSnowflake is most definitely running multiple instances for anything but the smallest clusters. It's plausible that one warehouse might scale better as you increase the number of nodes but I would really be surprised if there was a big difference---the techniques for query execution are pretty well-known, so Redshift and Snowflake are probably doing the same things. I did a benchmark I while ago [1] and I'm definitely planning on adding multiple scales to evaluate how they compare as you increase data size. [1] https://blog.fivetran.com/warehouse-benchmark-dce9f4c529c1 https://blog.fivetran.com/warehouse-benchmark-dce9f4c529c1
- MechanicalTwerk 9y agoI'm going to get downvoted to all hell but come on? All caps? Scales up forever?
- sol_remmy 9y agoI say unlimited because concurrent reads were basically a non-issue for Snowflake. You could have as many users reading from the same table as you wanted. Meanwhile "Redshift enforces a query concurrency limit of 15 on a cluster." http://blog.blazeclan.com/what-is-amazon-redshift-11-key-points-remember/ http://blog.blazeclan.com/what-is-amazon-redshift-11-key-poi...
- dang 9y agoCould you please don't use uppercase for emphasis? The site guidelines ask you not to: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. I'll mark your comment editable for a while if you want to correct this, and can delete this comment if you do.