17 ms·

Snap commits $2B over 5 years for Google Cloud infrastructure

341 points by samaysharma 10y ago

whalesalad 10y ago
If I could do it all over again I would probably opt for Google. The Kubernetes support is wonderful and the overall user experience blows AWS out of the water.
- toomuchtodo 10y ago
  Why opt for Google if you're going to use containers in Kubernetes? You then become cloud agnostic. You can even move to your own datacenter at some point (relatively) easily. Dropbox built out their own environment (and did it migrating 500PB out of S3) [1] [1a]. As did Twitter [2]. And Facebook [3]. And GitLab [4] (too soon?) As well as Mixpanel [5]. Even Twilio is multi-cloud (last time I checked it was split between AWS and Rackspace; this was several years ago during an interview, so maybe its changed). Sure, start in Google, or AWS, but at some point you will either need to use multiple compute/storage providers (redundancy) or go to your own gear (redundancy and cost). Example: "In 2014, Moz CEO Sarah Bird said that it was spending “$6.2 million at Amazon Web Services, and a mere $2.8 million on [its] own data centers.” Simply put, the cloud killed its margins." [6] EDIT: simonebrunozzi: Forgive me, but when you're talking about hundreds of millions of dollars in spend, "easy" is relative. It is much easier when you're not relying on underlying primitives that are difficult to reproduce on your own at another provider (witness how terrible Open Stack is; no one wants to do that if they don't have to). Am I minimizing the effort involved for this discussion? For sure. But the money involved...it solves most problems you would have migrating between providers. > It seems to me that you have no serious experience in the real world. You are entitled to your opinion. I have seen the pain, and it is relative. Its easier when someone says, "Here is the budget, just fix the problem", and your vendor's (AWS/Google) margins are 20-40% (these are real margins pulled from earnings reports); that's a lot of money you can put back in your own (or your shareholders') pockets. If you we're spending $2 billion dollars, and I told you I could save you $400 million by spending $100 million, wouldn't you take that deal? Even at $200 million, its a bargain! For less than what Snap is spending on Google cloud infrastructure, SpaceX built a rocket that can take a payload to orbit and return the first stage successfully (SpaceX has taken on ~$1.2 billion in funding over the last 14 years). Moving out of a cloud provider is comparatively hard? EDIT: Maybe this is a roundabout way to kick back to Google in order to get preferential treatment on the Ad network. It sure isn't a logical decision. EDIT 2: @ashayh: I'm not saying go back to good ol' bare metal. For $2 billion, you could build your own cloud provider out as an internal operation. The amount that's being spent on Google Cloud is egregious, and worse yet, common shares have no voting rights to push back against poor decisions like this. EDIT 3: @hueving: HN throttles my posting; editing this comment is my only way to respond. Sorry about that! [1] https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/ https://www.wired.com/2016/03/epic-story-dropboxs-exodus-ama... [1a] http://www.informationweek.com/cloud/cloud-storage/how-dropbox-moved-500pb-of-customer-files-off-aws/d/d-id/1325721 http://www.informationweek.com/cloud/cloud-storage/how-dropb... [2] https://blog.twitter.com/2016/overview-of-the-twitter-cloud-platform-compute https://blog.twitter.com/2016/overview-of-the-twitter-cloud-... [3] https://code.facebook.com/hardware/ https://code.facebook.com/hardware/ | http://www.zdnet.com/pictures/facebooks-data-centers-worldwide-by-the-numbers-and-in-pictures/ http://www.zdnet.com/pictures/facebooks-data-centers-worldwi... [4] https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ [5] https://code.mixpanel.com/2011/10/27/why-we-moved-off-the-cloud/ https://code.mixpanel.com/2011/10/27/why-we-moved-off-the-cl... [6] http://www.thewhir.com/blog/moving-away-from-aws-cloud-dropbox-isnt-an-anomaly-and-heres-why http://www.thewhir.com/blog/moving-away-from-aws-cloud-dropb...
  - Yodoshi 10y ago
    Because of google container engine (hosted kubernetes). https://cloud.google.com/container-engine/ https://cloud.google.com/container-engine/ Also them starting the project along with the knowledge they have internally scaling containers helps.
    - deleted 10y ago
      [deleted]
    - nikcub 10y ago
      Can anyone here provide some info or make a comparison with the Azure Container Service here, or any AWS option? I'm going to go through all of the service offerings this weekend - from Docker Inc's Docker for Azure and Docker for AWS to the native container services on each.
      - ryanl-ee 10y ago
        Azure Container Service is simply a PaaS-ish offering of Swam, DC/OS, or Kubernetes. It still spins up VMs that you can log into, but handles deployment/provisioning of the product, and makes some assumptions about your use case. It's a great way to push a button and have a "real" deployment of those services to evaluate, especially if your goal is a platform-agnostic target. I work for an Azure-focused cloud consultancy and for any serious production environment we still build out a more custom deployment using a combination of Terraform, Chef, Cloud-Init, CoreOS, etc.
        nikcub 10y ago
        Thanks for the comment - I'm going to try it out. Want to email me your consultancy (email in profile) - I work with a bunch of diff companies who use Azue
    - hatred 10y ago
      > Also them starting the project along with the knowledge they have internally scaling containers helps. FWIW, Most of this are advertising gimmicks though and Google has a pretty different internal infrastructure for orchestrating containers that has hardly much to do with K8s.
- hosh 10y ago
  I did do it all over again (at a different company) and tried Google. And it does blow AWS out of the water. I don't regret hand-installing Kubernetes on AWS, if only because I know a lot more about how it works now. But I rather just spin it up with GKE.
  - alex-mohr 10y ago
    Glad to hear GKE's working well for you -- thanks hosh and whalesalad! We do invest a lot in making Google Container Engine a great experience, including integrating it with other parts of GCP (e.g., IAM [1]), but at the same time, the core is plain vanilla Kubernetes. [1] https://cloud.google.com/container-engine/docs/iam-integration https://cloud.google.com/container-engine/docs/iam-integrati...
hemancuso 10y ago
So they hit $400m of revenue in 2016 and have committed to spend at least that much on infrastructure each year for the next 5 years? After all the costs for staffing and everything else they better I hope they achieve amazing growth if they ever intend to profit.
- gigatexal 10y ago
  It's a crazy world we live in that's for sure.
  - user5994461 10y ago
    There is surely a way to turn that into "accounting debt" that will cancel lots of taxes and save a lot of money one way or another.
    - harryh 10y ago
      I don't know what you mean by "accounting debt" but yes companies only pay taxes on profits. In addition, present day losses can cancel out future profits. There is nothing nefarious about this.
    - pbarnes_1 10y ago
      That's... just normal expenses. This is how things operate.
      - smallegan 10y ago
        It always boggles my mind how often I come across people that don't understand how our tax system works...
        grzm 10y ago
        Ha! Assuming we're discussing the US, I'm consistently amazed at how complicated it is. The number of tax accountants and wide success of TurboTax lend credence to the idea that it's more complicated than a lot of people want to deal with.
user5994461 10y ago
This makes no sense. One cannot spend $33M a month on Google Cloud. (Remember that it's half the price of AWS, and given a contract of that magnitude it's possible that they negotiated yet another half off). The amount of hardware and services one would get for that bill is insane. Snapchat doesn't need that much computing power and storage.
- bpicolo 10y ago
  Yeah, that seems really high. $400m a year? They're 10% of Google's yearly cloud revenue on their own? That said it's definitely what the s-1 filing claims: "Any transition of the cloud services currently provided by Google Cloud to another cloud provider would be difficult to implement and will cause us to incur significant time and expense. We have committed to spend $2 billion with Google Cloud over the next five years and have built our software and computer systems to use computing, storage capabilities, bandwidth, and other services provided by Google, some of which do not have an alternative in the market."
  - siong1987 10y ago
    Where do you get that 10% from? From the article... "Google doesn’t break out revenues from its cloud infrastructure, choosing to lump it in with other non-advertising businesses like hardware and Google Play sales. But that segment totaled $3.4 billion in sales in the most recent quarter." Assuming that all $3.4 billion is for cloud revenue last quarter, that is still less than 10% for last quarter ($400m/4 = $100m).
    - bpicolo 10y ago
      Hmm, might have been looking at q4 2015 which looked like 800 million a quarter. Still, a significant share either way yeah?
- freyir 10y ago
  Maybe $400M/year unlocks Google customer support?
deleted 10y ago
[deleted]
filereaper 10y ago
I'm confused by this high cost, sure you might need lots of compute, but storage tends to dominate over the long term. Isn't Snapchat supposed to delete data after 30s or whatever? Why this insanely high cost? Can somebody shed some light.
- teej 10y ago
  I imagine bandwidth is a major cost center.
  - amluto 10y ago
    Last time I checked, Google and AWS both had very, very high egress prices. At these levels, it's far cheaper to connect to a Tier 1 provider or three.
    - hadfgdasf 10y ago
      Did you check what their price was at the $2B purchase volume?
      - amluto 10y ago
        Nope. OTOH, $2B is large enough that the Tier 1 transit rates are probably quite good, too.
- argonaut 10y ago
  1. There are also stories (last 24 hours). These are often video clips. 2. Like Whatsapp, server has to store the message until the receiver is online again (until they open the app). 3. Whatever systems they use for advertising, tracking, profiling, and analytics is probably a significant chunk of their backend.
  - kevindong 10y ago
    > 2. Like Whatsapp, server has to store the message until the receiver is online again (until they open the app). The server actually has to store it for a maximum of 30 days [0]. Snaps get deleted after 30 days (even if they're unopened). They're deleted immediately once the Snapchat servers get confirmation they've been viewed. [0]: https://www.snap.com/en-US/news/post/how-snaps-are-stored-and-deleted/ https://www.snap.com/en-US/news/post/how-snaps-are-stored-an...
scottmcleod 10y ago
Nice Snapchat files to IPO so they can pay Google bill. Wonder who wins in this long game...
- dopamean 10y ago
  Google. I can half imagine a scenario where Snap fails to meet growth expectations and is bought on the cheap by google.
- eip 10y ago
  The CIA. As usual.
scottmcleod 10y ago
Nice Snapchat files to IPO so they can pay Google bill. Wonder who wins in this long game...
angryteabag 10y ago
Can I get someone's opinion on Snap? Is it worth paying attention to? My understanding is that it's a package manager that installs applications in their own isolated Linux sandbox, meaning you can install/distribute them on any distribution.. right? Does that mean software like node.js or nginx/apache will be available via Snap?
- dom2 10y ago
  I can't tell if you're joking but Snap is Snapchat. So not a package manager at all.
- niftich 10y ago
  I think you're confusing the company formerly known as Snapchat, now called 'Snap Inc.' with the 'Snappy' package manager (hosted out of 'snapcraft.io') which makes linuxy packages called 'snaps'. The two are unrelated.
  - angryteabag 10y ago
    I am getting confused and you nailed it. I'm referring to Snappy
  - krzyk 10y ago
    I also got confused (I was wondering "who is Snap, and why is he investing in Google"), Snap is not that popular at the news sites that I read (which is mostly Hacker News :), at least after their name change.
randartie 10y ago
Needing this amount of resources implies that Snap is expecting huge growth. This sounds like a really bad move on their part and they should have committed to building out their own infrastructure on 'bare metal' over the next 5 years instead. If you read their S-1, they list a dependence on Google cloud as one of their big risk factors. Yet they then go ahead and make this commitment instead of working towards eliminating it. There's so many advantages to owning your stack and if Snap thinks that it's going to need 2 billion dollars to pay for cloud infra, they're at the scale where it makes sense to build your own infra. Just look at Facebook, they're able to create tailor made data-centers that fit precisely what they need. The success of Snap relies on huge scale on the consumer side, if they want to scale their infra to support that 5 years down the line then this sounds like a poor move since they will either need to play catch-up later on or prepare to pay serious dough to Google. Paying for cloud services seems like a great idea when you are not able to predict your needs in the coming years, given a deal like this I don't think that's the case.
- paulddraper 10y ago
  Precisely. Google Cloud is a good idea when you don't know how much infrastructure you need. Of course, maybe they got some killer promotional deal with Google. Seems likely.
  - averageweather 10y ago
    How much did that sales rep get in commission? ;-)
  - chetanahuja 10y ago
    > Of course, maybe they got some killer promotional deal with Google For sure. How much is this free marketing that Google cloud service is getting worth? I'm pretty sure whatever discounted deal Google gave Snap is more than made up by this free marketing blitz they're getting.
    - theDoug 10y ago
      Snap’s been a happy and public customer for some time, so any “free marketing blitz” would a) have essentially been used up before, and b) would truly have to be remarkable to work against some form of discount where the non-discounted remainder /still/ represents $2,000,000,000 over four years.
jpeg_hero 10y ago
This could be a major issue with the ipo. Wall Street investors don't take kindly to long term commitments such as this. It looks especially impertinent because it is on the order of their current revenue ($400m per year due to google versus $404m rev 2016). The similarities in the numbers just beg comparison..
nikcub 10y ago
The timing is curious, they signed this dead and added $2B in fixed liabilities on their balance sheet days or weeks before the S1 filing. That they didn't wait until after the IPO suggests Snap may see the partnership as a positive. Could also have been pushed from Google's end as it is a nice way of bragging about a big get but without having to formally announce it, and being associated with a hot IPO that will get a lot of coverage.
- eitally 10y ago
  Or they could have been running on GCP for a long while and just didn't announce it previously.... (they initially built their product on AppEngine)
ErikAugust 10y ago
They started on App Engine, I believe? Interesting that just having one startup like Snapchat start on their platform made all the other little startups that came and went worth it.
deleted 10y ago
[deleted]
sobinator 10y ago
Many talking of software here, but at this scale I think we should be looking at the cost of energy. Suppose Google has a true edge on the rest of the market in terms of what the cost of a Watt is to them. Take that outlook over the horizon of 5 years; all software arguments are thereby moot. If Google can generate a Watt in 5 years at 10% the cost that AWS can, then this drastically changes the equation.
- smeyer 10y ago
  I don't follow these things closely, but what's the reason to expect that Google will be able to generate a watt at 10% of the cost that AWS can? 10x seems like a massively impressive edge to have over another huge player in the market for a commodity good like power.
  - sobinator 10y ago
    Good question. No reason to expect Google beating AWS, but absolutely Google beats Snap on price per Watt if Snap builds this infra out themselves. I'm no expert, but I'd say that when the outlook is five years and you're Google and you have your own internal energy hedge fund, a big margin (perhaps not 10x) is within the realm of possibility. https://en.wikipedia.org/wiki/Google_Energy https://en.wikipedia.org/wiki/Google_Energy
- specialist 10y ago
  Exactly. https://en.wikipedia.org/wiki/Performance_per_watt https://en.wikipedia.org/wiki/Performance_per_watt
beefsack 10y ago
For anyone else who was confused who Snap are, they do Snapchat (it wasn't immediately obvious to me.)
- plorntus 10y ago
  Hah yeah, to me too. Wasnt until half way through the comments I found out who Snap was. Could have at least used the former name in the article somewhere or their logo.
general_ai 10y ago
I wonder if they're still using Google AppEngine, or have moved to something lower level. GAE resolved a lot of its scalability and isolation issues thanks to Snapchat. In a way, Snapchat was to GAE what Hotmail was to Windows NT back in the day — trial by fire.
deleted 10y ago
[deleted]
tabeth 10y ago
What's the back-end equivalent of "free scaling?" For example, with the front-end, if you have a SPA or a website that's completely static, you can serve static files and JavaScript in a way that scales horizontally for free. Is there a back-end designed (with compromises and all) in a way where scaling horizontally is free, at the expense of compute power or some other sacrifice? I want to say Erlang/Elixir, but I haven't played around with it enough to say for sure.
- toong 10y ago
  Stateless applications are a big part of the answer. "Serverless" (like AWS Lambda) is an extreme impl of stateless that gets "free scaling". You will need to keep state somewhere and AWS has some pretty decent solutions for that.
- luckystartup 10y ago
  Most server backends can easily scale horizontally, including Ruby on Rails, Node.js, and Elixir. The only requirement is that you don't keep any "state" in your server code. A novice programmer can make this mistake in any language, including Elixir. Read about the "Twelve-Factor App": https://12factor.net/ https://12factor.net/ The main bottleneck is usually your database, which can be tricky to scale. You set up things like read slaves and sharding.
mastazi 10y ago
From the article: > Access to Google, which currently powers our infrastructure, is restricted in China. What does this mean exactly? I could read the sentence above in two ways: A- if a website is on Google infrastructure, then it will not be accessible in China B- if a website is on Google infrastructure, then the cloud control panel will not be accessible to the IT operations personnel based in China I think that it's the latter, I find it hard to believe that the scenario described at point A corresponds to reality. If that was the case it would have huge implications in terms of competition between Google and other providers.
- firloop 10y ago
  I think it may be possible due to sharing of Google infrastructure that Google App Engine apps (what Snapchat runs on) are automatically blocked in China. This trick allowed Whatsapp to spoof censorship recently by feigning a connection to Google.com while truly connecting to an App Engine hosted proxy.
- owenwil 10y ago
  It's part A - Google's cloud infrastructure isn't even resolvable in China.
kriro 10y ago
Makes me wonder if Google will buy a substantial amount of Snap stock and if there were takeover discussions before the IPO. I'm not sure if I like this from Snaps POV but it seems like they expect massive growth and possibly want to focus more on the core business than on infrastructure (which could come back to bite them strategically but I'm sure they got a very sweet deal).
eddd 10y ago
So 2.5$/year per user, I am not sure if that is a lot. How much does facebook spent on their infra per user basis?
- stp-ip 10y ago
  For current user volumes. Due to them filing for IPO it seems right to assume that they want to multiply their user count quite a bit.
krzyk 10y ago
For ones (like me) wondering what Snap is, it is the company behind Snapchat, they apparently changed their name few months ago (source: https://en.wikipedia.org/wiki/Snap_Inc. https://en.wikipedia.org/wiki/Snap_Inc.)
- andybak 10y ago
  Thank you. I was wondering the same.
sebleon 10y ago
What do you guys think Google Cloud's margins are like? Presumably, Google is able to run a data center more efficiently than Snap could, meaning that the savings of running their own data centers will be strictly less than GC's margins... thoughts?
deleted 10y ago
[deleted]
kartickv 10y ago
It seems dangerous to commit to spending so much money for so long into the future, with a particular vendor. Who knows whether Google Cloud will be best for your needs four years from now? Price, performance, reliability, support, whether it will have best-in-class abstractions, and so on...
deleted 10y ago
[deleted]
dhsbdb 10y ago
Spotify are also heavy users of Google Cloud: https://news.spotify.com/us/2016/02/23/announcing-spotify-infrastructures-googley-future/ https://news.spotify.com/us/2016/02/23/announcing-spotify-in...
anonu 10y ago
Snap Inc can negotiate this number down if their revenue targets aren't met. So its really just a pro-forma agreement that can be changed... Source: The S1... the article fails to mention that part.
johnsmith21006 10y ago
Google also owns a piece of Snap through their venture arm. Plus I would think Snap will want to stay close to Google for access to their advertising exchange. We are quickly getting down to just two advertising exchanges Google and Facebook.
010a 10y ago
It is so concerning to me that the company has committed to spending an average of $400M per year on cloud infra when their revenue is only $200M, and they've revealed that user growth slowed from 17% to 3% in the quarter Insta released Stories. It'd be one thing if they were going to use some of the IPO money to cost-optimize revenue, but I get the feeling that they need to focus on growing revenue due to how Insta Stories gutted them in 2016. That means hiring more people and writing bigger checks to Google. And they're branding themselves as a "camera company." Their hardware division does not contribute materially to revenue (not profit: Revenue), and practically every other consumer camera company, from Kodak to GoPro, is dying. This is not a healthy company.
fabbuki 10y ago
Snapchat could afford to build its own infrastructure if it wanted to. A similar sized Unicorn, DocuSign, has about 2000 employees and probably several hundred million in revenue per year (valuation is around a billion or two, depending on who you ask). They built their own data centers around the world. But DocuSign derives a lot of money from B2B and uptime and location of the servers is important to other Enterprises. DocuSign also started building out its services more than a decade ago before a lot of the AWS or Google Cloud infra got built. So, the decision to build your own infra is as much a decision based on alternatives available. Few alternatives? Build your own infra. By staying on GCP, Snapchat also guarantees its service and uptime will not change appreciably over the next several years. They built on GCP and migrating the whole service off would probably be a gargantuan task (how do you flip a switch and move all your compute overnight without hurting customer experience?). Staying with GCP allows Snap to maintain consistency of service while also buying time to build a transferable version of Snapchat that they could move to other infrastructure after the Google contract is over. Investors on Wallstreet don't like seeing huge changes to company strategy too close to IPO. If GCP has worked for Snap thus far, it is far less risky to investors for Snap to keep on going "business as usual." It's better to overspend to guarantee certainty of service and business health over next few years than do a massive capital investment. Once Snap gets off the ground post-IPO, they can make longer term decisions about their infra.

341 points by samaysharma 10y ago

If I could do it all over again I would probably opt for Google. The Kubernetes support is wonderful and the overall user experience blows AWS out of the water.

toomuchtodo 10y ago
Why opt for Google if you're going to use containers in Kubernetes? You then become cloud agnostic. You can even move to your own datacenter at some point (relatively) easily. Dropbox built out their own environment (and did it migrating 500PB out of S3) [1] [1a]. As did Twitter [2]. And Facebook [3]. And GitLab [4] (too soon?) As well as Mixpanel [5]. Even Twilio is multi-cloud (last time I checked it was split between AWS and Rackspace; this was several years ago during an interview, so maybe its changed). Sure, start in Google, or AWS, but at some point you will either need to use multiple compute/storage providers (redundancy) or go to your own gear (redundancy and cost). Example: "In 2014, Moz CEO Sarah Bird said that it was spending “$6.2 million at Amazon Web Services, and a mere $2.8 million on [its] own data centers.” Simply put, the cloud killed its margins." [6] EDIT: simonebrunozzi: Forgive me, but when you're talking about hundreds of millions of dollars in spend, "easy" is relative. It is much easier when you're not relying on underlying primitives that are difficult to reproduce on your own at another provider (witness how terrible Open Stack is; no one wants to do that if they don't have to). Am I minimizing the effort involved for this discussion? For sure. But the money involved...it solves most problems you would have migrating between providers. > It seems to me that you have no serious experience in the real world. You are entitled to your opinion. I have seen the pain, and it is relative. Its easier when someone says, "Here is the budget, just fix the problem", and your vendor's (AWS/Google) margins are 20-40% (these are real margins pulled from earnings reports); that's a lot of money you can put back in your own (or your shareholders') pockets. If you we're spending $2 billion dollars, and I told you I could save you $400 million by spending $100 million, wouldn't you take that deal? Even at $200 million, its a bargain! For less than what Snap is spending on Google cloud infrastructure, SpaceX built a rocket that can take a payload to orbit and return the first stage successfully (SpaceX has taken on ~$1.2 billion in funding over the last 14 years). Moving out of a cloud provider is comparatively hard? EDIT: Maybe this is a roundabout way to kick back to Google in order to get preferential treatment on the Ad network. It sure isn't a logical decision. EDIT 2: @ashayh: I'm not saying go back to good ol' bare metal. For $2 billion, you could build your own cloud provider out as an internal operation. The amount that's being spent on Google Cloud is egregious, and worse yet, common shares have no voting rights to push back against poor decisions like this. EDIT 3: @hueving: HN throttles my posting; editing this comment is my only way to respond. Sorry about that! [1] https://www.wired.com/2016/03/epic-story-dropboxs-exodus-amazon-cloud-empire/ https://www.wired.com/2016/03/epic-story-dropboxs-exodus-ama... [1a] http://www.informationweek.com/cloud/cloud-storage/how-dropbox-moved-500pb-of-customer-files-off-aws/d/d-id/1325721 http://www.informationweek.com/cloud/cloud-storage/how-dropb... [2] https://blog.twitter.com/2016/overview-of-the-twitter-cloud-platform-compute https://blog.twitter.com/2016/overview-of-the-twitter-cloud-... [3] https://code.facebook.com/hardware/ https://code.facebook.com/hardware/ | http://www.zdnet.com/pictures/facebooks-data-centers-worldwide-by-the-numbers-and-in-pictures/ http://www.zdnet.com/pictures/facebooks-data-centers-worldwi... [4] https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ https://about.gitlab.com/2016/11/10/why-choose-bare-metal/ [5] https://code.mixpanel.com/2011/10/27/why-we-moved-off-the-cloud/ https://code.mixpanel.com/2011/10/27/why-we-moved-off-the-cl... [6] http://www.thewhir.com/blog/moving-away-from-aws-cloud-dropbox-isnt-an-anomaly-and-heres-why http://www.thewhir.com/blog/moving-away-from-aws-cloud-dropb...
- Yodoshi 10y ago
  Because of google container engine (hosted kubernetes). https://cloud.google.com/container-engine/ https://cloud.google.com/container-engine/ Also them starting the project along with the knowledge they have internally scaling containers helps.
  - deleted 10y ago
    [deleted]
  - nikcub 10y ago
    Can anyone here provide some info or make a comparison with the Azure Container Service here, or any AWS option? I'm going to go through all of the service offerings this weekend - from Docker Inc's Docker for Azure and Docker for AWS to the native container services on each.
    - ryanl-ee 10y ago
      Azure Container Service is simply a PaaS-ish offering of Swam, DC/OS, or Kubernetes. It still spins up VMs that you can log into, but handles deployment/provisioning of the product, and makes some assumptions about your use case. It's a great way to push a button and have a "real" deployment of those services to evaluate, especially if your goal is a platform-agnostic target. I work for an Azure-focused cloud consultancy and for any serious production environment we still build out a more custom deployment using a combination of Terraform, Chef, Cloud-Init, CoreOS, etc.
      - nikcub 10y ago
        Thanks for the comment - I'm going to try it out. Want to email me your consultancy (email in profile) - I work with a bunch of diff companies who use Azue
  - hatred 10y ago
    > Also them starting the project along with the knowledge they have internally scaling containers helps. FWIW, Most of this are advertising gimmicks though and Google has a pretty different internal infrastructure for orchestrating containers that has hardly much to do with K8s.
hosh 10y ago
I did do it all over again (at a different company) and tried Google. And it does blow AWS out of the water. I don't regret hand-installing Kubernetes on AWS, if only because I know a lot more about how it works now. But I rather just spin it up with GKE.
- alex-mohr 10y ago
  Glad to hear GKE's working well for you -- thanks hosh and whalesalad! We do invest a lot in making Google Container Engine a great experience, including integrating it with other parts of GCP (e.g., IAM [1]), but at the same time, the core is plain vanilla Kubernetes. [1] https://cloud.google.com/container-engine/docs/iam-integration https://cloud.google.com/container-engine/docs/iam-integrati...

hemancuso 10y ago

So they hit $400m of revenue in 2016 and have committed to spend at least that much on infrastructure each year for the next 5 years? After all the costs for staffing and everything else they better I hope they achieve amazing growth if they ever intend to profit.

gigatexal 10y ago
It's a crazy world we live in that's for sure.
- user5994461 10y ago
  There is surely a way to turn that into "accounting debt" that will cancel lots of taxes and save a lot of money one way or another.
  - harryh 10y ago
    I don't know what you mean by "accounting debt" but yes companies only pay taxes on profits. In addition, present day losses can cancel out future profits. There is nothing nefarious about this.
  - pbarnes_1 10y ago
    That's... just normal expenses. This is how things operate.
    - smallegan 10y ago
      It always boggles my mind how often I come across people that don't understand how our tax system works...
      - grzm 10y ago
        Ha! Assuming we're discussing the US, I'm consistently amazed at how complicated it is. The number of tax accountants and wide success of TurboTax lend credence to the idea that it's more complicated than a lot of people want to deal with.

user5994461 10y ago

This makes no sense. One cannot spend $33M a month on Google Cloud. (Remember that it's half the price of AWS, and given a contract of that magnitude it's possible that they negotiated yet another half off). The amount of hardware and services one would get for that bill is insane. Snapchat doesn't need that much computing power and storage.

bpicolo 10y ago
Yeah, that seems really high. $400m a year? They're 10% of Google's yearly cloud revenue on their own? That said it's definitely what the s-1 filing claims: "Any transition of the cloud services currently provided by Google Cloud to another cloud provider would be difficult to implement and will cause us to incur significant time and expense. We have committed to spend $2 billion with Google Cloud over the next five years and have built our software and computer systems to use computing, storage capabilities, bandwidth, and other services provided by Google, some of which do not have an alternative in the market."
- siong1987 10y ago
  Where do you get that 10% from? From the article... "Google doesn’t break out revenues from its cloud infrastructure, choosing to lump it in with other non-advertising businesses like hardware and Google Play sales. But that segment totaled $3.4 billion in sales in the most recent quarter." Assuming that all $3.4 billion is for cloud revenue last quarter, that is still less than 10% for last quarter ($400m/4 = $100m).
  - bpicolo 10y ago
    Hmm, might have been looking at q4 2015 which looked like 800 million a quarter. Still, a significant share either way yeah?
freyir 10y ago
Maybe $400M/year unlocks Google customer support?

deleted 10y ago

[deleted]

filereaper 10y ago

I'm confused by this high cost, sure you might need lots of compute, but storage tends to dominate over the long term. Isn't Snapchat supposed to delete data after 30s or whatever? Why this insanely high cost? Can somebody shed some light.

teej 10y ago
I imagine bandwidth is a major cost center.
- amluto 10y ago
  Last time I checked, Google and AWS both had very, very high egress prices. At these levels, it's far cheaper to connect to a Tier 1 provider or three.
  - hadfgdasf 10y ago
    Did you check what their price was at the $2B purchase volume?
    - amluto 10y ago
      Nope. OTOH, $2B is large enough that the Tier 1 transit rates are probably quite good, too.
argonaut 10y ago
1. There are also stories (last 24 hours). These are often video clips. 2. Like Whatsapp, server has to store the message until the receiver is online again (until they open the app). 3. Whatever systems they use for advertising, tracking, profiling, and analytics is probably a significant chunk of their backend.
- kevindong 10y ago
  > 2. Like Whatsapp, server has to store the message until the receiver is online again (until they open the app). The server actually has to store it for a maximum of 30 days [0]. Snaps get deleted after 30 days (even if they're unopened). They're deleted immediately once the Snapchat servers get confirmation they've been viewed. [0]: https://www.snap.com/en-US/news/post/how-snaps-are-stored-and-deleted/ https://www.snap.com/en-US/news/post/how-snaps-are-stored-an...

scottmcleod 10y ago

Nice Snapchat files to IPO so they can pay Google bill. Wonder who wins in this long game...

dopamean 10y ago
Google. I can half imagine a scenario where Snap fails to meet growth expectations and is bought on the cheap by google.
eip 10y ago
The CIA. As usual.

scottmcleod 10y ago

Nice Snapchat files to IPO so they can pay Google bill. Wonder who wins in this long game...

angryteabag 10y ago

Can I get someone's opinion on Snap? Is it worth paying attention to? My understanding is that it's a package manager that installs applications in their own isolated Linux sandbox, meaning you can install/distribute them on any distribution.. right? Does that mean software like node.js or nginx/apache will be available via Snap?

dom2 10y ago
I can't tell if you're joking but Snap is Snapchat. So not a package manager at all.
niftich 10y ago
I think you're confusing the company formerly known as Snapchat, now called 'Snap Inc.' with the 'Snappy' package manager (hosted out of 'snapcraft.io') which makes linuxy packages called 'snaps'. The two are unrelated.
- angryteabag 10y ago
  I am getting confused and you nailed it. I'm referring to Snappy
- krzyk 10y ago
  I also got confused (I was wondering "who is Snap, and why is he investing in Google"), Snap is not that popular at the news sites that I read (which is mostly Hacker News :), at least after their name change.

randartie 10y ago

Needing this amount of resources implies that Snap is expecting huge growth. This sounds like a really bad move on their part and they should have committed to building out their own infrastructure on 'bare metal' over the next 5 years instead. If you read their S-1, they list a dependence on Google cloud as one of their big risk factors. Yet they then go ahead and make this commitment instead of working towards eliminating it. There's so many advantages to owning your stack and if Snap thinks that it's going to need 2 billion dollars to pay for cloud infra, they're at the scale where it makes sense to build your own infra. Just look at Facebook, they're able to create tailor made data-centers that fit precisely what they need. The success of Snap relies on huge scale on the consumer side, if they want to scale their infra to support that 5 years down the line then this sounds like a poor move since they will either need to play catch-up later on or prepare to pay serious dough to Google. Paying for cloud services seems like a great idea when you are not able to predict your needs in the coming years, given a deal like this I don't think that's the case.

paulddraper 10y ago
Precisely. Google Cloud is a good idea when you don't know how much infrastructure you need. Of course, maybe they got some killer promotional deal with Google. Seems likely.
- averageweather 10y ago
  How much did that sales rep get in commission? ;-)
- chetanahuja 10y ago
  > Of course, maybe they got some killer promotional deal with Google For sure. How much is this free marketing that Google cloud service is getting worth? I'm pretty sure whatever discounted deal Google gave Snap is more than made up by this free marketing blitz they're getting.
  - theDoug 10y ago
    Snap’s been a happy and public customer for some time, so any “free marketing blitz” would a) have essentially been used up before, and b) would truly have to be remarkable to work against some form of discount where the non-discounted remainder /still/ represents $2,000,000,000 over four years.

jpeg_hero 10y ago

This could be a major issue with the ipo. Wall Street investors don't take kindly to long term commitments such as this. It looks especially impertinent because it is on the order of their current revenue ($400m per year due to google versus $404m rev 2016). The similarities in the numbers just beg comparison..

nikcub 10y ago

The timing is curious, they signed this dead and added $2B in fixed liabilities on their balance sheet days or weeks before the S1 filing. That they didn't wait until after the IPO suggests Snap may see the partnership as a positive. Could also have been pushed from Google's end as it is a nice way of bragging about a big get but without having to formally announce it, and being associated with a hot IPO that will get a lot of coverage.

eitally 10y ago
Or they could have been running on GCP for a long while and just didn't announce it previously.... (they initially built their product on AppEngine)

ErikAugust 10y ago

They started on App Engine, I believe? Interesting that just having one startup like Snapchat start on their platform made all the other little startups that came and went worth it.

deleted 10y ago

[deleted]

sobinator 10y ago

Many talking of software here, but at this scale I think we should be looking at the cost of energy. Suppose Google has a true edge on the rest of the market in terms of what the cost of a Watt is to them. Take that outlook over the horizon of 5 years; all software arguments are thereby moot. If Google can generate a Watt in 5 years at 10% the cost that AWS can, then this drastically changes the equation.

smeyer 10y ago
I don't follow these things closely, but what's the reason to expect that Google will be able to generate a watt at 10% of the cost that AWS can? 10x seems like a massively impressive edge to have over another huge player in the market for a commodity good like power.
- sobinator 10y ago
  Good question. No reason to expect Google beating AWS, but absolutely Google beats Snap on price per Watt if Snap builds this infra out themselves. I'm no expert, but I'd say that when the outlook is five years and you're Google and you have your own internal energy hedge fund, a big margin (perhaps not 10x) is within the realm of possibility. https://en.wikipedia.org/wiki/Google_Energy https://en.wikipedia.org/wiki/Google_Energy
specialist 10y ago
Exactly. https://en.wikipedia.org/wiki/Performance_per_watt https://en.wikipedia.org/wiki/Performance_per_watt

beefsack 10y ago

For anyone else who was confused who Snap are, they do Snapchat (it wasn't immediately obvious to me.)

plorntus 10y ago
Hah yeah, to me too. Wasnt until half way through the comments I found out who Snap was. Could have at least used the former name in the article somewhere or their logo.

general_ai 10y ago

I wonder if they're still using Google AppEngine, or have moved to something lower level. GAE resolved a lot of its scalability and isolation issues thanks to Snapchat. In a way, Snapchat was to GAE what Hotmail was to Windows NT back in the day — trial by fire.

deleted 10y ago

[deleted]

tabeth 10y ago

What's the back-end equivalent of "free scaling?" For example, with the front-end, if you have a SPA or a website that's completely static, you can serve static files and JavaScript in a way that scales horizontally for free. Is there a back-end designed (with compromises and all) in a way where scaling horizontally is free, at the expense of compute power or some other sacrifice? I want to say Erlang/Elixir, but I haven't played around with it enough to say for sure.

toong 10y ago
Stateless applications are a big part of the answer. "Serverless" (like AWS Lambda) is an extreme impl of stateless that gets "free scaling". You will need to keep state somewhere and AWS has some pretty decent solutions for that.
luckystartup 10y ago
Most server backends can easily scale horizontally, including Ruby on Rails, Node.js, and Elixir. The only requirement is that you don't keep any "state" in your server code. A novice programmer can make this mistake in any language, including Elixir. Read about the "Twelve-Factor App": https://12factor.net/ https://12factor.net/ The main bottleneck is usually your database, which can be tricky to scale. You set up things like read slaves and sharding.

mastazi 10y ago

From the article: > Access to Google, which currently powers our infrastructure, is restricted in China. What does this mean exactly? I could read the sentence above in two ways: A- if a website is on Google infrastructure, then it will not be accessible in China B- if a website is on Google infrastructure, then the cloud control panel will not be accessible to the IT operations personnel based in China I think that it's the latter, I find it hard to believe that the scenario described at point A corresponds to reality. If that was the case it would have huge implications in terms of competition between Google and other providers.

firloop 10y ago
I think it may be possible due to sharing of Google infrastructure that Google App Engine apps (what Snapchat runs on) are automatically blocked in China. This trick allowed Whatsapp to spoof censorship recently by feigning a connection to Google.com while truly connecting to an App Engine hosted proxy.
owenwil 10y ago
It's part A - Google's cloud infrastructure isn't even resolvable in China.

kriro 10y ago

Makes me wonder if Google will buy a substantial amount of Snap stock and if there were takeover discussions before the IPO. I'm not sure if I like this from Snaps POV but it seems like they expect massive growth and possibly want to focus more on the core business than on infrastructure (which could come back to bite them strategically but I'm sure they got a very sweet deal).

eddd 10y ago

So 2.5$/year per user, I am not sure if that is a lot. How much does facebook spent on their infra per user basis?

stp-ip 10y ago
For current user volumes. Due to them filing for IPO it seems right to assume that they want to multiply their user count quite a bit.

krzyk 10y ago

For ones (like me) wondering what Snap is, it is the company behind Snapchat, they apparently changed their name few months ago (source: https://en.wikipedia.org/wiki/Snap_Inc. https://en.wikipedia.org/wiki/Snap_Inc.)

andybak 10y ago
Thank you. I was wondering the same.

sebleon 10y ago

What do you guys think Google Cloud's margins are like? Presumably, Google is able to run a data center more efficiently than Snap could, meaning that the savings of running their own data centers will be strictly less than GC's margins... thoughts?

deleted 10y ago

[deleted]

kartickv 10y ago

It seems dangerous to commit to spending so much money for so long into the future, with a particular vendor. Who knows whether Google Cloud will be best for your needs four years from now? Price, performance, reliability, support, whether it will have best-in-class abstractions, and so on...

deleted 10y ago

[deleted]

dhsbdb 10y ago

Spotify are also heavy users of Google Cloud: https://news.spotify.com/us/2016/02/23/announcing-spotify-infrastructures-googley-future/ https://news.spotify.com/us/2016/02/23/announcing-spotify-in...

anonu 10y ago

Snap Inc can negotiate this number down if their revenue targets aren't met. So its really just a pro-forma agreement that can be changed... Source: The S1... the article fails to mention that part.

johnsmith21006 10y ago

Google also owns a piece of Snap through their venture arm. Plus I would think Snap will want to stay close to Google for access to their advertising exchange. We are quickly getting down to just two advertising exchanges Google and Facebook.

010a 10y ago

It is so concerning to me that the company has committed to spending an average of $400M per year on cloud infra when their revenue is only $200M, and they've revealed that user growth slowed from 17% to 3% in the quarter Insta released Stories. It'd be one thing if they were going to use some of the IPO money to cost-optimize revenue, but I get the feeling that they need to focus on growing revenue due to how Insta Stories gutted them in 2016. That means hiring more people and writing bigger checks to Google. And they're branding themselves as a "camera company." Their hardware division does not contribute materially to revenue (not profit: Revenue), and practically every other consumer camera company, from Kodak to GoPro, is dying. This is not a healthy company.

fabbuki 10y ago

Snapchat could afford to build its own infrastructure if it wanted to. A similar sized Unicorn, DocuSign, has about 2000 employees and probably several hundred million in revenue per year (valuation is around a billion or two, depending on who you ask). They built their own data centers around the world. But DocuSign derives a lot of money from B2B and uptime and location of the servers is important to other Enterprises. DocuSign also started building out its services more than a decade ago before a lot of the AWS or Google Cloud infra got built. So, the decision to build your own infra is as much a decision based on alternatives available. Few alternatives? Build your own infra. By staying on GCP, Snapchat also guarantees its service and uptime will not change appreciably over the next several years. They built on GCP and migrating the whole service off would probably be a gargantuan task (how do you flip a switch and move all your compute overnight without hurting customer experience?). Staying with GCP allows Snap to maintain consistency of service while also buying time to build a transferable version of Snapchat that they could move to other infrastructure after the Google contract is over. Investors on Wallstreet don't like seeing huge changes to company strategy too close to IPO. If GCP has worked for Snap thus far, it is far less risky to investors for Snap to keep on going "business as usual." It's better to overspend to guarantee certainty of service and business health over next few years than do a massive capital investment. Once Snap gets off the ground post-IPO, they can make longer term decisions about their infra.