4 ms·

Regardless of the long time horizon, it is still a significant number. Cloud is known to not be the cheapest option at large scale, so it is a bit curious they

Regardless of the long time horizon, it is still a significant number. Cloud is known to not be the cheapest option at large scale, so it is a bit curious they decided not to build their own infra. I'm sure the folks at Pinterest are smart cookies, so this is an indicator that cloud is now cost competitive even at scale (with these kinds of deals, which presumably come with a discount).

taylodl 5y ago
> "Cloud is known to not be the cheapest option at large scale" Oh really? Ever try building, staffing, powering, cooling and equipping a data center? And then build a DR? Let me tell you, my company has foolishly done that just in the past ten years and WE are screaming uncle! We're now transitioning to AWS because it's much, much, much cheaper! The biggest eye-opener? Not only is the physical part of the data center a super expensive ongoing expense - staff, server life cycles, power, property tax - but the software licenses (we gotta have support!) will EAT YOU ALIVE! I've transitioned many projects to AWS and was able to justify it on the ongoing licensing costs alone - never mind all the other costs related to building a data center. I bet Pinterest did their homework and realized the $3 billion they're going to pay to AWS is cheaper than building, staffing, and running their own infrastructure. That's the conclusion we came to - after building our own data centers.
- jjav 5y ago
  > Oh really? Ever try building, staffing, powering, cooling and equipping a data center? And then build a DR? Let me tell you, my company has foolishly done that just in the past ten years and WE are screaming uncle! We're now transitioning to AWS because it's much, much, much cheaper! Since most of my career postdates and predates renting capacity on someone else's computer, yes, have done that many times. Would be great to publish a case study on your experience, would love to understand the numbers. While for some rare scenarios AWS can be cheaper, for just about any use case it's going to be much more expensive than running your own.
- fooblat 5y ago
  I did exactly this project for a high traffic site about 10 years ago. We got datacenter rooms wholesale that included power and cooling. Did everything else ourselves and cut our burn rate in half. However, we were totally open source so no licensing costs. Even after leasing all the hardware, hiring the staff, getting the transit brought in, we still cut our compute costs in half. Even so, we ended up keeping cloudfront+S3 for serving static content because we couldn't beat the price they offered.
  - CodesInChaos 5y ago
    I'm surprised that static content is where AWS is competitive for you, considering the high egress pricing ($20/TB is the lowest advertised price for cloudfront).
    - fooblat 5y ago
      It is highly negotiable when you get to large volumes. We got a term where CloudFront fetches from EC2 were free for objects <101K (or something like that) and free for EC2 from S3. We were able to take great advantage of that with the way we built our app. It was a long while ago so I may be off on the exact details.
  - harpratap 5y ago
    > However, we were totally open source so no licensing costs. This is not a healthy opinion to have, not for the open source community and neither for the company using open source projects. All the bugs and missing features of the open source project you use are also a part of your technical debt now, you can either donate to project to keep it healthy or hire FTE working on contributing to these projects, which is just indirect form of payment and should absolutely be a part of your budget
    - fooblat 5y ago
      You seem to be jumping to a lot of conclusions simply because we chose freely available open source. To have a business depend on software you either need a support service or you support it yourself. We chose the latter and made sure the team had the skills to do so. Of course we contributed our bug fixes back and even released our own tools open source.
robjan 5y ago
When you become a significant user of cloud they start to offer pretty significant negotiated discounts on top of committed usage discounts.
jasode 5y ago
>Cloud is known to not be the cheapest option at large scale, so it is a bit curious they decided not to build their own infra. The "cloud" provided by AWS is cheaper -- if you use the higher-level AWS portfolio of services . E.g. DynamoDB, Redshift, Kinesis, etc. If the mental model of "cloud" is just "bare metal servers", aka EC2, then AWS is more expensive than self-owned datacenters. So yes, if you're only looking at cpu and egress costs, AWS as a "dumb data center" is a terrible deal. But that's not how Pinterest uses AWS. They use AWS high-level services: E.g. they use AWS Rekognition -- machine learning service to annotate images with text : https://aws.amazon.com/solutions/case-studies/innovators/pinterest/ https://aws.amazon.com/solutions/case-studies/innovators/pin... Pinterest also tried self-hosted Elasticsearch but it cost them more money than AWS Elasticseaerch: https://aws.amazon.com/solutions/case-studies/pinterest-elasticsearch-case-study/ https://aws.amazon.com/solutions/case-studies/pinterest-elas... (Yes, those are AWS marketing articles but writing those case studies require cooperation and approval by the customer (Pinterest in this case.).) So far, Pinterest concluded it would not be cheaper to build their own machine learning algorithms to process images. Paying for AWS Rekognition is part of the ~$400m/year cost.
- brutus1213 5y ago
  Interesting points. Is AWS Rekognition just doing OCR as a service? I guess there isn't enough value in that to build a competitive solution. So good point. For more complex perceptual models, I'd be really surprised if the model wasn't considered proprietary/value-added.

taylodl 5y ago

> "Cloud is known to not be the cheapest option at large scale" Oh really? Ever try building, staffing, powering, cooling and equipping a data center? And then build a DR? Let me tell you, my company has foolishly done that just in the past ten years and WE are screaming uncle! We're now transitioning to AWS because it's much, much, much cheaper! The biggest eye-opener? Not only is the physical part of the data center a super expensive ongoing expense - staff, server life cycles, power, property tax - but the software licenses (we gotta have support!) will EAT YOU ALIVE! I've transitioned many projects to AWS and was able to justify it on the ongoing licensing costs alone - never mind all the other costs related to building a data center. I bet Pinterest did their homework and realized the $3 billion they're going to pay to AWS is cheaper than building, staffing, and running their own infrastructure. That's the conclusion we came to - after building our own data centers.

jjav 5y ago
> Oh really? Ever try building, staffing, powering, cooling and equipping a data center? And then build a DR? Let me tell you, my company has foolishly done that just in the past ten years and WE are screaming uncle! We're now transitioning to AWS because it's much, much, much cheaper! Since most of my career postdates and predates renting capacity on someone else's computer, yes, have done that many times. Would be great to publish a case study on your experience, would love to understand the numbers. While for some rare scenarios AWS can be cheaper, for just about any use case it's going to be much more expensive than running your own.
fooblat 5y ago
I did exactly this project for a high traffic site about 10 years ago. We got datacenter rooms wholesale that included power and cooling. Did everything else ourselves and cut our burn rate in half. However, we were totally open source so no licensing costs. Even after leasing all the hardware, hiring the staff, getting the transit brought in, we still cut our compute costs in half. Even so, we ended up keeping cloudfront+S3 for serving static content because we couldn't beat the price they offered.
- CodesInChaos 5y ago
  I'm surprised that static content is where AWS is competitive for you, considering the high egress pricing ($20/TB is the lowest advertised price for cloudfront).
  - fooblat 5y ago
    It is highly negotiable when you get to large volumes. We got a term where CloudFront fetches from EC2 were free for objects <101K (or something like that) and free for EC2 from S3. We were able to take great advantage of that with the way we built our app. It was a long while ago so I may be off on the exact details.
- harpratap 5y ago
  > However, we were totally open source so no licensing costs. This is not a healthy opinion to have, not for the open source community and neither for the company using open source projects. All the bugs and missing features of the open source project you use are also a part of your technical debt now, you can either donate to project to keep it healthy or hire FTE working on contributing to these projects, which is just indirect form of payment and should absolutely be a part of your budget
  - fooblat 5y ago
    You seem to be jumping to a lot of conclusions simply because we chose freely available open source. To have a business depend on software you either need a support service or you support it yourself. We chose the latter and made sure the team had the skills to do so. Of course we contributed our bug fixes back and even released our own tools open source.

robjan 5y ago

When you become a significant user of cloud they start to offer pretty significant negotiated discounts on top of committed usage discounts.

jasode 5y ago

>Cloud is known to not be the cheapest option at large scale, so it is a bit curious they decided not to build their own infra. The "cloud" provided by AWS is cheaper -- if you use the higher-level AWS portfolio of services . E.g. DynamoDB, Redshift, Kinesis, etc. If the mental model of "cloud" is just "bare metal servers", aka EC2, then AWS is more expensive than self-owned datacenters. So yes, if you're only looking at cpu and egress costs, AWS as a "dumb data center" is a terrible deal. But that's not how Pinterest uses AWS. They use AWS high-level services: E.g. they use AWS Rekognition -- machine learning service to annotate images with text : https://aws.amazon.com/solutions/case-studies/innovators/pinterest/ https://aws.amazon.com/solutions/case-studies/innovators/pin... Pinterest also tried self-hosted Elasticsearch but it cost them more money than AWS Elasticseaerch: https://aws.amazon.com/solutions/case-studies/pinterest-elasticsearch-case-study/ https://aws.amazon.com/solutions/case-studies/pinterest-elas... (Yes, those are AWS marketing articles but writing those case studies require cooperation and approval by the customer (Pinterest in this case.).) So far, Pinterest concluded it would not be cheaper to build their own machine learning algorithms to process images. Paying for AWS Rekognition is part of the ~$400m/year cost.

brutus1213 5y ago
Interesting points. Is AWS Rekognition just doing OCR as a service? I guess there isn't enough value in that to build a competitive solution. So good point. For more complex perceptual models, I'd be really surprised if the model wasn't considered proprietary/value-added.