5 ms·

"won many others. Kinesis vs. Kafka, DocumentDB vs. MongoDB, MemoryDB vs. Redis, OpenSearch vs. ElasticSearch." Lol. Wrong on all of those.

by vm2196 5y ago

FridgeSeal 5y ago
I was just about to add a comment about this too. Also saying they “missed the Snowflake opportunity” doesn’t make sense either: they have Redshift? They don’t need to acquire Snowflake, they’re already the incumbent in the field.
- rckrd 5y ago
  Author here. They missed the Snowflake opportunity by having the wrong architecture for Redshift (decoupled storage and compute). They shifted to the Snowflake model in 2019, but the damage might already be done. For other the other services I listed, the main differentiators are mainly plugins/extensibility and developer experience.
  - FridgeSeal 5y ago
    Isn’t this a bit of a moot point now though? If you’re going to pay someone for a data warehouse, you can pay AWS some money, or you can pay Snowflake a heinous amount of money, for something not significantly better? Mongo, ES and Redis all come with open source versions, which shifts the appeal back away from the AWS offerings, especially with things like Elastic Cloud for Kubernetes which has made our ES cluster basically a hands free experience.
    - glogla 5y ago
      Snowflake is technically far superior to Redshift. The performance and features are somewhere else. Even if you pay more in operating cost (which is not always the case, especially if you count Redshift DBA salary), you get to do what you actually need to do much faster instead of fighting the platform. Snowflake is also technically going to always be far superior to Redshift, because AWS is a follower, not a leader. Their strategy is to copy what others are doing and build a moat of "but we can do it too, and you already have other things with us". And if you really care about operating cost, you should be running Trino+Minio or Clickhouse onprem or something as your warehouse.
      - FridgeSeal 5y ago
        > Snowflake is technically far superior to Redshift. The performance and features are somewhere else From the discussions I’ve had about this before, I think I’m in the minority when I say I’m categorically unimpressed by Snowflakes performance. Add that to the hideous cost, and the worlds most aggressive sales/account management team and I’ve less than zero desire to ever deal with them again. > Snowflake is also technically going to always be far superior to Redshift, because AWS is a follower While this is true, how many businesses actually need or actually utilise the features they’re paying for with things like Snowflake. Sure it’s got separate storage and compute, but how many places have so much data that they need that? I’ve worked with places that were running multi-node data warehouse clusters, that we migrated to a single ClickHouse or Postgres instance and got equivalent or better performance for a fraction of the economic and operational overhead. > And if you really care about operating cost, you should be running Trino+Minio or Clickhouse onprem or something as your warehouse. Can’t disagree here, although I’d say it shouldn’t just be relegated to operating cost: CH blows most alternatives out of the water and is available in hosted options now. Haven’t used Trino recently, last time I used it, it was still called Presto, and it was frustratingly slow, know if it’s improved recently?
        glogla 5y ago
        I think that depends on your target and especially scope. ClickHouse demonstrates this well - it is incredibly fast and powerful, but also very limited. It has its own dialect of SQL incompatible with anything else. It doesn't even have a traditional query planner so you have to be expert to write fast queries. It has no update, no merge, no CTEs. To get most out of it, you have to think about data sorting, data types (is this LowCardinality String or just String?). And once you reach scale where single node can handle it (which is much more than single node Redshift could handle) you manage a stateful cluster which is a pain compared to storage/compute separated clusters like Presto/Trino over S3. So ClickHouse is very good if it is in hands of dedicated team that is willing to learn CH in and out and is using it for a particular purpose. But you can't really setup ClickHouse, dump data into it and then send 100 random Data Analysts to "go forth and make reports". The barrier of entry and list of things that are different in ClickHouse is just too long. Snowflake is exactly at the opposite end of "how much knowledge / how much of a dedicated team do I need to make use of this". Dumping data in and giving other teams access is something you can do easily. Scaling isn't a problem, users interfering with each other isn't a problem (you can give them their own compute that suspends when not in use), all the typical "data features" are there, backup is handled via time travel once they delete something they shouldn't, they don't have to understand data types or what is the best partitioning and sorting for the querying they are planning to do. It even has fancy web UI with charts and CSV upload. You do pay a pretty penny for that, but there are many companies with a lot of money and many data problems to solve, so paying more for infrastructure and enabling a many of less technically skilled users (who are likely SMEs) is worth it. Redshift sits somewhere in the middle. You still need DBAs, you still somewhat need to think about data layout, you still need to worry about managing backups and workload management where one using can slow down everyone else, etc. It feels like worst of all worlds. In onprem world, Trino behaves more like Snowflake than ClickHouse, which has its uses - again if youre looking to build a general platform instead of dedicated application. Might be worth it even if you never get milisecond latency queries out of it.
  - vm2196 5y ago
    I don't know what you are smoking here but let's take MongoDB vs DocumentDB or Elastic Cloud. Mongo built a 30B$ business around Atlas that is growing at a staggering rate. DocumentDB has a tiny share. Same with Elastic. Also what the hell are you talking about with RedShift. Decoupled Compute and Storage is a far superior architecture. Here is an expert from snowflake.com - " "Snowflake was founded on the belief that tying compute and storage together is not an effective approach for limitless, seamless scaling."
  - hodgesrm 5y ago
    Are you saying AWS does not make money off the infrastructure Snowflake consumes? Snowflake consumes a lot of compute, plus there are lots of other SaaS services that integrate with it and also need compute, networking, etc. It's my understanding the margins on compute at least are pretty good. So it seems as if AWS wins either way.
tstrimple 5y ago
Yeah, I'm not sure the point they are trying to make here. Each of these AWS services is clearly and demonstrably less popular than the alternative it's compared against. Mongo and DocumentDB for example aren't even close: https://db-engines.com/en/ranking_trend/system/Amazon+DocumentDB%3BMongoDB https://db-engines.com/en/ranking_trend/system/Amazon+Docume...
- ljm 5y ago
  OpenSearch is a massively inferior offering compared to Elasticsearch too. It became outdated the moment it was forked, the documentation is lacking, and since you'll end up looking up ES docs and forgetting to switch to version 7.10, you'll get a nice reminder of everything new that has been added that you can't actually use.
- lmeyerov 5y ago
  For big cloud markets, being #2 or #3 is a win: - #2, #3, even maybe #4/#5 are big revenue. Cloud is unusually big and still growing insanely. - crazy margins when they don't have to invent the core concept, go through core product/market fit R&D, nor fight for a distribution channel to market+sell it, nor fight middlemen for competitive pricing - cross-selling & ecosystem lock-in means even revenue/profit don't have to be high or even positive
- ljm 5y ago
  OpenSearch is a massively inferior offering compared to Elasticsearch too. It became outdated the moment it was forked, the documentation is lacking, and since you'll end up looking up ES docs and forgetting to switch to version 7.10, you'll get a nice reminder of everything new that has been added that you can't actually use. The only thing it has going for it is that it's managed and you're already on AWS, so you don't need to spend months working up a contract with a new vendor and doing the security audit dance.
  - noogle 5y ago
    > so you don't need to spend months working up a contract with a new vendor and doing the security audit dance. That's a very big moat. Many decision makers are risk-averse w.r.t to infrastructure vendors and don't mind paying (or making someone else pay) a premium for that. The only thing that changed in the saying "no one was fire for choosing IBM" is the name.

"won many others. Kinesis vs. Kafka, DocumentDB vs. MongoDB, MemoryDB vs. Redis, OpenSearch vs. ElasticSearch." Lol. Wrong on all of those.

by vm2196 5y ago

"won many others. Kinesis vs. Kafka, DocumentDB vs. MongoDB, MemoryDB vs. Redis, OpenSearch vs. ElasticSearch." Lol. Wrong on all of those.

FridgeSeal 5y ago

I was just about to add a comment about this too. Also saying they “missed the Snowflake opportunity” doesn’t make sense either: they have Redshift? They don’t need to acquire Snowflake, they’re already the incumbent in the field.

rckrd 5y ago
Author here. They missed the Snowflake opportunity by having the wrong architecture for Redshift (decoupled storage and compute). They shifted to the Snowflake model in 2019, but the damage might already be done. For other the other services I listed, the main differentiators are mainly plugins/extensibility and developer experience.
- FridgeSeal 5y ago
  Isn’t this a bit of a moot point now though? If you’re going to pay someone for a data warehouse, you can pay AWS some money, or you can pay Snowflake a heinous amount of money, for something not significantly better? Mongo, ES and Redis all come with open source versions, which shifts the appeal back away from the AWS offerings, especially with things like Elastic Cloud for Kubernetes which has made our ES cluster basically a hands free experience.
  - glogla 5y ago
    Snowflake is technically far superior to Redshift. The performance and features are somewhere else. Even if you pay more in operating cost (which is not always the case, especially if you count Redshift DBA salary), you get to do what you actually need to do much faster instead of fighting the platform. Snowflake is also technically going to always be far superior to Redshift, because AWS is a follower, not a leader. Their strategy is to copy what others are doing and build a moat of "but we can do it too, and you already have other things with us". And if you really care about operating cost, you should be running Trino+Minio or Clickhouse onprem or something as your warehouse.
    - FridgeSeal 5y ago
      > Snowflake is technically far superior to Redshift. The performance and features are somewhere else From the discussions I’ve had about this before, I think I’m in the minority when I say I’m categorically unimpressed by Snowflakes performance. Add that to the hideous cost, and the worlds most aggressive sales/account management team and I’ve less than zero desire to ever deal with them again. > Snowflake is also technically going to always be far superior to Redshift, because AWS is a follower While this is true, how many businesses actually need or actually utilise the features they’re paying for with things like Snowflake. Sure it’s got separate storage and compute, but how many places have so much data that they need that? I’ve worked with places that were running multi-node data warehouse clusters, that we migrated to a single ClickHouse or Postgres instance and got equivalent or better performance for a fraction of the economic and operational overhead. > And if you really care about operating cost, you should be running Trino+Minio or Clickhouse onprem or something as your warehouse. Can’t disagree here, although I’d say it shouldn’t just be relegated to operating cost: CH blows most alternatives out of the water and is available in hosted options now. Haven’t used Trino recently, last time I used it, it was still called Presto, and it was frustratingly slow, know if it’s improved recently?
      - glogla 5y ago
        I think that depends on your target and especially scope. ClickHouse demonstrates this well - it is incredibly fast and powerful, but also very limited. It has its own dialect of SQL incompatible with anything else. It doesn't even have a traditional query planner so you have to be expert to write fast queries. It has no update, no merge, no CTEs. To get most out of it, you have to think about data sorting, data types (is this LowCardinality String or just String?). And once you reach scale where single node can handle it (which is much more than single node Redshift could handle) you manage a stateful cluster which is a pain compared to storage/compute separated clusters like Presto/Trino over S3. So ClickHouse is very good if it is in hands of dedicated team that is willing to learn CH in and out and is using it for a particular purpose. But you can't really setup ClickHouse, dump data into it and then send 100 random Data Analysts to "go forth and make reports". The barrier of entry and list of things that are different in ClickHouse is just too long. Snowflake is exactly at the opposite end of "how much knowledge / how much of a dedicated team do I need to make use of this". Dumping data in and giving other teams access is something you can do easily. Scaling isn't a problem, users interfering with each other isn't a problem (you can give them their own compute that suspends when not in use), all the typical "data features" are there, backup is handled via time travel once they delete something they shouldn't, they don't have to understand data types or what is the best partitioning and sorting for the querying they are planning to do. It even has fancy web UI with charts and CSV upload. You do pay a pretty penny for that, but there are many companies with a lot of money and many data problems to solve, so paying more for infrastructure and enabling a many of less technically skilled users (who are likely SMEs) is worth it. Redshift sits somewhere in the middle. You still need DBAs, you still somewhat need to think about data layout, you still need to worry about managing backups and workload management where one using can slow down everyone else, etc. It feels like worst of all worlds. In onprem world, Trino behaves more like Snowflake than ClickHouse, which has its uses - again if youre looking to build a general platform instead of dedicated application. Might be worth it even if you never get milisecond latency queries out of it.
- vm2196 5y ago
  I don't know what you are smoking here but let's take MongoDB vs DocumentDB or Elastic Cloud. Mongo built a 30B$ business around Atlas that is growing at a staggering rate. DocumentDB has a tiny share. Same with Elastic. Also what the hell are you talking about with RedShift. Decoupled Compute and Storage is a far superior architecture. Here is an expert from snowflake.com - " "Snowflake was founded on the belief that tying compute and storage together is not an effective approach for limitless, seamless scaling."
- hodgesrm 5y ago
  Are you saying AWS does not make money off the infrastructure Snowflake consumes? Snowflake consumes a lot of compute, plus there are lots of other SaaS services that integrate with it and also need compute, networking, etc. It's my understanding the margins on compute at least are pretty good. So it seems as if AWS wins either way.

tstrimple 5y ago

Yeah, I'm not sure the point they are trying to make here. Each of these AWS services is clearly and demonstrably less popular than the alternative it's compared against. Mongo and DocumentDB for example aren't even close: https://db-engines.com/en/ranking_trend/system/Amazon+DocumentDB%3BMongoDB https://db-engines.com/en/ranking_trend/system/Amazon+Docume...

ljm 5y ago
OpenSearch is a massively inferior offering compared to Elasticsearch too. It became outdated the moment it was forked, the documentation is lacking, and since you'll end up looking up ES docs and forgetting to switch to version 7.10, you'll get a nice reminder of everything new that has been added that you can't actually use.
lmeyerov 5y ago
For big cloud markets, being #2 or #3 is a win: - #2, #3, even maybe #4/#5 are big revenue. Cloud is unusually big and still growing insanely. - crazy margins when they don't have to invent the core concept, go through core product/market fit R&D, nor fight for a distribution channel to market+sell it, nor fight middlemen for competitive pricing - cross-selling & ecosystem lock-in means even revenue/profit don't have to be high or even positive
ljm 5y ago
OpenSearch is a massively inferior offering compared to Elasticsearch too. It became outdated the moment it was forked, the documentation is lacking, and since you'll end up looking up ES docs and forgetting to switch to version 7.10, you'll get a nice reminder of everything new that has been added that you can't actually use. The only thing it has going for it is that it's managed and you're already on AWS, so you don't need to spend months working up a contract with a new vendor and doing the security audit dance.
- noogle 5y ago
  > so you don't need to spend months working up a contract with a new vendor and doing the security audit dance. That's a very big moat. Many decision makers are risk-averse w.r.t to infrastructure vendors and don't mind paying (or making someone else pay) a premium for that. The only thing that changed in the saying "no one was fire for choosing IBM" is the name.