7 ms·

The issue here is that it's quite impossible to compete with AWS if it offers even an inferior version of a service. Because of network egress pricing. You ar

The issue here is that it's quite impossible to compete with AWS if it offers even an inferior version of a service. Because of network egress pricing. You are most likely on AWS already. And even if elastic offers a 2x better product at 1/2 the price, your cost of traffic for an external service will easily 4x versus using AWS services. And that's really the frustration for these service providers. You can NEVER compete. Ever. Against an equivalent service offered by AWS because the egress pricing acts as a pricing barrier.

pradn 6y ago
And there's not a way to remove the egress pricing moat because it's so natural. Of course it costs less money to move data within a datacenter than outside of it. So, why wouldn't it cost less to do everything inside AWS? Egress pricing is probably multiples of this actual discrepancy to erect the moat even deeper. It really does feel hopeless to run an open-source service that isn't running on an existing crowd. Perhaps that's one reason Snowflake is doing well.
- boulos 6y ago
  See my sibling comment. Snowflake actually has historically run as a multinenant service with access to say your S3 bucket, so the majority of data transfer is VM <=> S3 in the same Region (so included/free). As a snowflake customer, you would submit queries (and data ingestion) as Egress pricing (roughly $.01/GB though assuming it was using the private link or similar) and then the smallish results come back as Ingress (which AWS charges for, we do not, but whatever). I hope someone corrects me on the AWS side (I really thought that there was a way to do zero rating for same “AZ” third-party services), but it’s not fundamental.
  - sandGorgon 6y ago
    no this is not free. Because what you would do is setup a "peering" between Snowflake cluster and your own. Even if it is the same region, there is a cost - https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-basics.html https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-b... That cost acts as a barrier if AWS launches its own Snowflake competitor. Basically AWS taxes you for not using other AWS services, once you are inside AWS.
  - pradn 6y ago
    This doesn't get around the egress pricing for sending data from AWS to another company's data center, yes?
boulos 6y ago
Disclosure: I work on Google Cloud. Just to clarify, AWS has a feature called PrivateLink [1] that charges $.01/GB transferred between a Service provider and the end customer (I thought there was a free variant for same AZ, but I seem to be wrong about that). We offer a similar feature called VPC Peering, and in particular Private Service Access [2]. Because we can be sure that the traffic is in the same Zone (or not), talking to a third-party service in the same Zone does not incur egress charges. But did you mean this $.01/GB or were you thinking of VM <=> Internet / External IP pricing? [1] https://aws.amazon.com/privatelink/ https://aws.amazon.com/privatelink/ [2] https://cloud.google.com/vpc/docs/configure-private-services-access https://cloud.google.com/vpc/docs/configure-private-services...
- sandGorgon 6y ago
  Generally a VM <> external pricing. Its tricky to setup Privatelink with a service provider. For example Elastic cloud will have its own kubernetes setup. It could be behind a HW loadbalancer - how do you set up Privatelink there ? Super tricky and not applicable in all circumstances. GCP and AWS must move towards sane traffic pricing (or risk anti-monopoly lawsuits). The licensing changes are just a first step. P.S. : offtopic... can i use privatelink between AWS and Google Cloud ? do you have an idea on how much it would cost on either side ?
  - BillinghamJ 6y ago
    I don't think it would be hard to offer PrivateLink to tenants via Kubernetes. You'd just make a K8s LoadBalancer, set the annotations to back it with an NLB, then expose that with PrivateLink.
  - boulos 6y ago
    > can i use privatelink between AWS and Google Cloud ? do you have an idea on how much it would cost on either side ? You basically have to hookup Cloud Provider A through a middle layer to Cloud Provider B (e.g., connect via Equinix or Megaport). It’s not drastically different than a hybrid setup, except you don’t control the premises on either side. While getting you the right link to our Partner Interconnect docs, it turns out we have a nice solution guide writing up the various options [1]. The costs boil down to some non-trivial fixed price each month to cover the 1/10/100+ Gbps connection on each side, plus whatever transfer fees each provider charges for egress over that method. At large enough volumes, providers of all sorts usually have equivalent fixed prices. Getting multiple redundant lines, or doing VPN encryption, obviously costs more and is more work than “just a pipe”. Even at list prices though, this gets you closer to $.02/GB each way in Europe or North America. At high volume in the same metro area (like Northern Virginia), you can easily agree to much lower rates on either side. [1] https://cloud.google.com/solutions/patterns-for-connecting-other-csps-with-gcp https://cloud.google.com/solutions/patterns-for-connecting-o...
hodgesrm 6y ago
What's the basis of your statement? For data warehouse/data lake type applications, data typically goes in and aggregates come out. The point of these systems is to fish out only the data you actually need to make decisions, which is usually much smaller than the source dataset. Compute and storage therefore tend to dwarf other costs.
- sandGorgon 6y ago
  we are talking elasticsearch here. This is not aggregate computation. you are constantly pushing data to ES and retrieving queries.
ayush--s 6y ago
This is quite true. Right now we are working to move our $(msg queue software) from $(one of its market leaders) to AWS's in-house offering, because the network transfer cost is twice the cost of our subscription on the $(one of its market leaders).
kenturamon 6y ago
Elastic Cloud offers VPC endpoint connections if your deployment is in AWS. You don't end up paying the usual NAT gateway egress costs.

pradn 6y ago

And there's not a way to remove the egress pricing moat because it's so natural. Of course it costs less money to move data within a datacenter than outside of it. So, why wouldn't it cost less to do everything inside AWS? Egress pricing is probably multiples of this actual discrepancy to erect the moat even deeper. It really does feel hopeless to run an open-source service that isn't running on an existing crowd. Perhaps that's one reason Snowflake is doing well.

boulos 6y ago
See my sibling comment. Snowflake actually has historically run as a multinenant service with access to say your S3 bucket, so the majority of data transfer is VM <=> S3 in the same Region (so included/free). As a snowflake customer, you would submit queries (and data ingestion) as Egress pricing (roughly $.01/GB though assuming it was using the private link or similar) and then the smallish results come back as Ingress (which AWS charges for, we do not, but whatever). I hope someone corrects me on the AWS side (I really thought that there was a way to do zero rating for same “AZ” third-party services), but it’s not fundamental.
- sandGorgon 6y ago
  no this is not free. Because what you would do is setup a "peering" between Snowflake cluster and your own. Even if it is the same region, there is a cost - https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-basics.html https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-b... That cost acts as a barrier if AWS launches its own Snowflake competitor. Basically AWS taxes you for not using other AWS services, once you are inside AWS.
- pradn 6y ago
  This doesn't get around the egress pricing for sending data from AWS to another company's data center, yes?

boulos 6y ago

Disclosure: I work on Google Cloud. Just to clarify, AWS has a feature called PrivateLink [1] that charges $.01/GB transferred between a Service provider and the end customer (I thought there was a free variant for same AZ, but I seem to be wrong about that). We offer a similar feature called VPC Peering, and in particular Private Service Access [2]. Because we can be sure that the traffic is in the same Zone (or not), talking to a third-party service in the same Zone does not incur egress charges. But did you mean this $.01/GB or were you thinking of VM <=> Internet / External IP pricing? [1] https://aws.amazon.com/privatelink/ https://aws.amazon.com/privatelink/ [2] https://cloud.google.com/vpc/docs/configure-private-services-access https://cloud.google.com/vpc/docs/configure-private-services...

sandGorgon 6y ago
Generally a VM <> external pricing. Its tricky to setup Privatelink with a service provider. For example Elastic cloud will have its own kubernetes setup. It could be behind a HW loadbalancer - how do you set up Privatelink there ? Super tricky and not applicable in all circumstances. GCP and AWS must move towards sane traffic pricing (or risk anti-monopoly lawsuits). The licensing changes are just a first step. P.S. : offtopic... can i use privatelink between AWS and Google Cloud ? do you have an idea on how much it would cost on either side ?
- BillinghamJ 6y ago
  I don't think it would be hard to offer PrivateLink to tenants via Kubernetes. You'd just make a K8s LoadBalancer, set the annotations to back it with an NLB, then expose that with PrivateLink.
- boulos 6y ago
  > can i use privatelink between AWS and Google Cloud ? do you have an idea on how much it would cost on either side ? You basically have to hookup Cloud Provider A through a middle layer to Cloud Provider B (e.g., connect via Equinix or Megaport). It’s not drastically different than a hybrid setup, except you don’t control the premises on either side. While getting you the right link to our Partner Interconnect docs, it turns out we have a nice solution guide writing up the various options [1]. The costs boil down to some non-trivial fixed price each month to cover the 1/10/100+ Gbps connection on each side, plus whatever transfer fees each provider charges for egress over that method. At large enough volumes, providers of all sorts usually have equivalent fixed prices. Getting multiple redundant lines, or doing VPN encryption, obviously costs more and is more work than “just a pipe”. Even at list prices though, this gets you closer to $.02/GB each way in Europe or North America. At high volume in the same metro area (like Northern Virginia), you can easily agree to much lower rates on either side. [1] https://cloud.google.com/solutions/patterns-for-connecting-other-csps-with-gcp https://cloud.google.com/solutions/patterns-for-connecting-o...

hodgesrm 6y ago

What's the basis of your statement? For data warehouse/data lake type applications, data typically goes in and aggregates come out. The point of these systems is to fish out only the data you actually need to make decisions, which is usually much smaller than the source dataset. Compute and storage therefore tend to dwarf other costs.

sandGorgon 6y ago
we are talking elasticsearch here. This is not aggregate computation. you are constantly pushing data to ES and retrieving queries.

ayush--s 6y ago

This is quite true. Right now we are working to move our $(msg queue software) from $(one of its market leaders) to AWS's in-house offering, because the network transfer cost is twice the cost of our subscription on the $(one of its market leaders).

kenturamon 6y ago

Elastic Cloud offers VPC endpoint connections if your deployment is in AWS. You don't end up paying the usual NAT gateway egress costs.