10 ms·

Etsy Announces Move to Google Cloud

171 points by xref 9y ago

merb 9y ago
wasn't etsy the company that was using amazon and shifted back to their own infra because they said cloud is not as cheap as own infra?
- illumin8 9y ago
  No, they've always self-hosted on bare metal.
  - stfwn 9y ago
    What's the difference between self-hosting and self-hosting on bare metal?
    - Someone1234 9y ago
      Self-hosted could mean running on a rented server managed by a vendor (e.g. Rackspace) often marketed as a "private cloud." The servers they're using are individually assigned to that client, the vendor manages the physical, the company manages every other layers. Self-hosted on bare mental usually means the company owns the physical servers, storage arrays, and so on. No renting or leasing. No subscription fee-like service. They may still be renting space in someone else's data center, but it is their hardware. The distinction isn't super important to be honest.
- ktta 9y ago
  It was Dropbox
clintonb 9y ago
Did they use another provider prior to the move, or self-host?
- whitepoplar 9y ago
  Last I checked, they hosted their own bare metal machines.
  - jasonlotito 9y ago
    Correct. I've seen their cages.
  - akgerber 9y ago
    In a data center in Jersey, at least a few years back.
- deleted 9y ago
  [deleted]
codingdave 9y ago
> Etsy's transition to the Google Cloud Platform is expected to be complete within the next two years. I have no idea what Etsy looks like internally, but that sounds like an awfully long time in this day and age. Anyone have an inkling as to why this is a multi-year project?
- tty7 9y ago
  someone set their DNS TTL to ∞
  - Something1234 9y ago
    Where are you going with this?
    - jeffbarr 9y ago
      To infinity and beyond, obviously....
    - mkhalil 9y ago
      He saying that is the reason why it's going to take years; because their DNS names will be pointing to the same IPs forever and not move over to the cloud based servers. It would of been funnier if you put a finite number, like TTL 3 years.
- sjeanpierre 9y ago
  I'd imagine that the move would require some re-engineering of their services to make the best use of GCP. I'd also imagine that their would be a non-trivial migration process to ensure uptime is maintained and customers are not impacted during the move.
- sitharus 9y ago
  Having worked for two companies that have shifted to cloud providers, multi-year timescales is fairly usual even just for like-for-like migrations. You can’t just shut down the servers, image them and start them in the cloud, you need to have a plan to progressively migrate services while maintaining site operation.
deleted 9y ago
[deleted]
foobarbazetc 9y ago
I love GCP but honestly have no idea how these big internet properties can make the $ case for “the cloud”, unless they’re also planning to fire 90% of the Ops/SysAdmin team (ala Evernote). We run on bare metal and to move to GCP would triple our hosting cost, not including bandwidth (which is free* for us). Do people get discounts for doing PR? I’m really curious.
- merb 9y ago
  most cost cuts come from automation (as you already noticed). but not just more automation in ops/sysadmin space, but mostly because you can just free resources you don't need. especially one-off jobs (analytics, etc.) take a huge amount of resources for a very short amount of time and after that time the resources aren't needed. on clouds you can hope to schedule them preemtivly. also most cost savings come at night, if you are not active on the whole globe you probably only need half of your resources at night. even if your global the chances are high that at some time you still need way less resources. etsy is a marketplace on most marketplaces there are times where demand is really high and demand is certainly not so high. (christmas, black firday, regressions, etc..) also if you self host, high availability can and is a pain. monitoring your database is a pain, it takes a huge amount of time, which most companies do not have. p.s. working at a small company and services like RDS/gcloud postgres! (thanks google) google buckets (not sure how they are called their name is not as cool as s3)/S3 are a huge win for us.
  - khc 9y ago
    "google buckets"/Google S3 is called GCS: Google Cloud Storage https://cloud.google.com/storage/ https://cloud.google.com/storage/ Aside: strangely enough googling GCS gives me Glasgow Coma Scale, you'd think by now google would have personalized the result for me
  - ThrustVectoring 9y ago
    Honest question: why not use both? If spinning up cloud computing resources is the most cost-efficient way to do those computing tasks, why not have that be the route for doing one-off computing tasks while your own machines run the workload for recurring stuff? Is there overhead for not having 100% of your computing done on a cloud platform that I'm not aware of or something?
vpol 9y ago
I’m not even sure Etsy will exist in two years.
- cracell 9y ago
  eBay has made way worst choices for a decade and it's still around.
- Someone1234 9y ago
  Based on this alone? Etsy seems to be fine for the most part. The only criticism I have of Etsy is that they've moved away or diluted their core business (small business/home business sales) by allowing mass manufactured goods to be sold on the site. Etsy still has a place, and I've ordered several items from Etsy vendors this last year. They just need to keep a check on what they want to be otherwise they'll be subpar eBay clone.
gregmac 9y ago
Found some interesting contrasting statements from 2015 [1]: > The company performed big data jobs early on in the cloud, using Amazon's Elastic Map Reduce service. > “That [EMR] got us to a place where we think about what data we collect. At the time it was really inefficient from a cost perspective, but there weren’t any creative constraints,” > The team eventually decided to bring it in house. “Our Hadoop cluster wields a huge amount of flexibility," said Rembetsy. "We can get a lot done on bare metal, and we know very much what we’re looking for. Cloud services, that’s where efficiencies fall away. Spin up and spin down doesn’t have the same value proposition.” [1] http://www.datacenterknowledge.com/archives/2015/05/11/how-etsy-optimized-data-center-infrastructure-for-efficiency http://www.datacenterknowledge.com/archives/2015/05/11/how-e...
- interskh 9y ago
  EMR is really expensive, especially you are using spot instances. It is roughly $2 for the EMR tax on $1 instance
josephhurtado 9y ago
It begs the question why Google Cloud and not MS Azure or Amazon's AWS? Could it be a price break they got, and also the AI advantages of Google Cloud's offerings? I am sure that played a big part.
- oh-kumudo 9y ago
  Probably because Google extends really really good deal to them for the PR effect. I don't think Etsy should consider itself as a competitor to Amazon, they simply can't. On the other side, AWS already has so many success stories that they are unlikely cater to Etsy specifically. That makes GCE the best option out there. As to Azure, I think people trust Google more as to cloud-based technology than Microsoft.
  - bpicolo 9y ago
    > they simply can't Amazon is one of Etsy's most relevant competitors. They'd be insane not to consider them a competitor https://www.amazon.com/Handmade https://www.amazon.com/Handmade
- bckygldstn 9y ago
  Many retail companies avoid AWS because they see Amazon as a competitor. Source: work for a retail company, we went with Azure.
  - TheReveller 9y ago
    I think that certainly comes into it. Seems perverse to take the money you make from sales and give a cut of it to your competitor, who can then reduce the prices in their own store.
    - billmalarky 9y ago
      That and there's a trust issue I'm sure, regarding hosting all your data with your competitor.
- 9y ago
code_duck 9y ago
Sounds good. I was an early Etsy seller, as a glass artist and bead vendor, As well as a developer who worked with them as closely as I could for a few years. They have always seemed a bit disconnected from their customers. The first regime, Rob Kalin, Haim and Revolving Dork, had an age and culture disconnect from most of their customers. The next group, headed by Chad Dickerson and Allspaw, seemed like they wanted to focus on technology for technology’s sake, more than what Etsy was actually doing with that as a company. They even made some statements to the effect of how they wanted Etsy to be known as a technology company, like Facebook, and be a platform, like Amazon. All fine and good, but so far, what all of the leaders of Etsy have had in common is that they don’t really understand what it’s like to be a micro business person selling handmade art online. If I was an investor, I wouldn’t want to be paying for a bunch of guys to experiment with Mongo DB search engines and shards. As long as they’re still not understanding their core business, it’s a distraction. Etsy should be focused on one thing: connecting buyers and sellers of handmade art with each other.
- BlanketApple 9y ago
  Disclaimer: I worked at Etsy, a long time ago. > The next group, headed by Chad Dickerson and Allspaw, seemed like they wanted to focus on technology for technology’s sake, more than what Etsy was actually doing with that as a company. I think you're missing quite a bit of this story. I was around during the transition of Chad from CTO -> CEO, and the truth is that Etsy needed to focus on the technology badly during that time. The site would go down constantly. We had very little insight into what was going on with sellers or buyers. Every busy day was basically us crossing our fingers that the site wouldn't go down. We needed someone focused heavily on the technology. We could never have continued on that path and grown as much as we did. > Etsy should be focused on one thing: connecting buyers and sellers of handmade art with each other. Deciding what potential buyers are interested in and what items to display to them is a pretty difficult technology problem. Detecting mass-produced goods in a way that's more subtle than "ban all Chinese sellers" is a pretty difficult technology problem. If anything I would argue that Etsy didn't focus nearly enough on that, and focused way too much on "low-hanging fruit" (minor workflow adjustments for sellers and buyers).
stareatgoats 9y ago
Not much meat for the technically interested here unfortunately. The respective business departments were clearly in charge of this press release. Which was a business mistake in my opinion: these press releases are primarily interesting for the CTOs and similar - the fluff presented here would serve to remind them of how they are kept at arms length i many board rooms even in their areas of expertise, more than anything. My guess is both Google Cloud and Etsy would dip in their opinion as result of it.
damm 9y ago
Okay so how much did they get for free by moving to Google Cloud? That's the question you should be asking. Etsy has been hurting; why would they move infrastructure now unless t hey got kind of kickback for the PR? This isn't an easy move; so there has to be a huge benefit to them.
CaptSpify 9y ago
Another DC to cloud story! I love these! It's like watching a play in 3 acts: Act 1) Yay, we did it, we're gonna save so much money! Act 2) [a year later] Wait, we're spending how much money?!? Act 3) It's going to cost us how much to leave our current cloud?!? Haven't people figured out yet that cloud companies make a profit? Sure, companies like Etsy can get a huge discount, but everyone else who isn't a big name is going to pay out the ass.
- btian 9y ago
  That's not how any of this works. Boeing makes a profit, but how much money will it cost to design and make a plane from scratch?
  - CaptSpify 9y ago
    > That's not how any of this works. Weird then, considering that I've seen >5 companies go through that process. Something that you say doesn't work one way clearly does. Planes are vastly different than data-centers. Depending on what you need, a brand new data-center can run as little as a couple hundred dollars for a server + parts, and <$30 / month on everything else. What is the cheapest plane you can buy?
_Codemonkeyism 9y ago
I would love to see a cost analysis of a company moving to GCP/AWS. We were a 6M customer flash sale company sold for $200M. We had 3 devops engineers mostly helping developers with designing the system and pager duty - mainly because 24/7 was to demanding for only 2 people. We were SOX/PCI compliant. Hosting costs all in all by year were around the same as the salaries I'd say. People who moved to AWS/GCP, would I save costs in this scenario moving to AWS/GCP?

171 points by xref 9y ago

merb 9y ago

wasn't etsy the company that was using amazon and shifted back to their own infra because they said cloud is not as cheap as own infra?

illumin8 9y ago
No, they've always self-hosted on bare metal.
- stfwn 9y ago
  What's the difference between self-hosting and self-hosting on bare metal?
  - Someone1234 9y ago
    Self-hosted could mean running on a rented server managed by a vendor (e.g. Rackspace) often marketed as a "private cloud." The servers they're using are individually assigned to that client, the vendor manages the physical, the company manages every other layers. Self-hosted on bare mental usually means the company owns the physical servers, storage arrays, and so on. No renting or leasing. No subscription fee-like service. They may still be renting space in someone else's data center, but it is their hardware. The distinction isn't super important to be honest.
ktta 9y ago
It was Dropbox

clintonb 9y ago

Did they use another provider prior to the move, or self-host?

whitepoplar 9y ago
Last I checked, they hosted their own bare metal machines.
- jasonlotito 9y ago
  Correct. I've seen their cages.
- akgerber 9y ago
  In a data center in Jersey, at least a few years back.
deleted 9y ago
[deleted]

codingdave 9y ago

> Etsy's transition to the Google Cloud Platform is expected to be complete within the next two years. I have no idea what Etsy looks like internally, but that sounds like an awfully long time in this day and age. Anyone have an inkling as to why this is a multi-year project?

tty7 9y ago
someone set their DNS TTL to ∞
- Something1234 9y ago
  Where are you going with this?
  - jeffbarr 9y ago
    To infinity and beyond, obviously....
  - mkhalil 9y ago
    He saying that is the reason why it's going to take years; because their DNS names will be pointing to the same IPs forever and not move over to the cloud based servers. It would of been funnier if you put a finite number, like TTL 3 years.
sjeanpierre 9y ago
I'd imagine that the move would require some re-engineering of their services to make the best use of GCP. I'd also imagine that their would be a non-trivial migration process to ensure uptime is maintained and customers are not impacted during the move.
sitharus 9y ago
Having worked for two companies that have shifted to cloud providers, multi-year timescales is fairly usual even just for like-for-like migrations. You can’t just shut down the servers, image them and start them in the cloud, you need to have a plan to progressively migrate services while maintaining site operation.

deleted 9y ago

[deleted]

foobarbazetc 9y ago

I love GCP but honestly have no idea how these big internet properties can make the $ case for “the cloud”, unless they’re also planning to fire 90% of the Ops/SysAdmin team (ala Evernote). We run on bare metal and to move to GCP would triple our hosting cost, not including bandwidth (which is free* for us). Do people get discounts for doing PR? I’m really curious.

merb 9y ago
most cost cuts come from automation (as you already noticed). but not just more automation in ops/sysadmin space, but mostly because you can just free resources you don't need. especially one-off jobs (analytics, etc.) take a huge amount of resources for a very short amount of time and after that time the resources aren't needed. on clouds you can hope to schedule them preemtivly. also most cost savings come at night, if you are not active on the whole globe you probably only need half of your resources at night. even if your global the chances are high that at some time you still need way less resources. etsy is a marketplace on most marketplaces there are times where demand is really high and demand is certainly not so high. (christmas, black firday, regressions, etc..) also if you self host, high availability can and is a pain. monitoring your database is a pain, it takes a huge amount of time, which most companies do not have. p.s. working at a small company and services like RDS/gcloud postgres! (thanks google) google buckets (not sure how they are called their name is not as cool as s3)/S3 are a huge win for us.
- khc 9y ago
  "google buckets"/Google S3 is called GCS: Google Cloud Storage https://cloud.google.com/storage/ https://cloud.google.com/storage/ Aside: strangely enough googling GCS gives me Glasgow Coma Scale, you'd think by now google would have personalized the result for me
- ThrustVectoring 9y ago
  Honest question: why not use both? If spinning up cloud computing resources is the most cost-efficient way to do those computing tasks, why not have that be the route for doing one-off computing tasks while your own machines run the workload for recurring stuff? Is there overhead for not having 100% of your computing done on a cloud platform that I'm not aware of or something?

vpol 9y ago

I’m not even sure Etsy will exist in two years.

cracell 9y ago
eBay has made way worst choices for a decade and it's still around.
Someone1234 9y ago
Based on this alone? Etsy seems to be fine for the most part. The only criticism I have of Etsy is that they've moved away or diluted their core business (small business/home business sales) by allowing mass manufactured goods to be sold on the site. Etsy still has a place, and I've ordered several items from Etsy vendors this last year. They just need to keep a check on what they want to be otherwise they'll be subpar eBay clone.

gregmac 9y ago

Found some interesting contrasting statements from 2015 [1]: > The company performed big data jobs early on in the cloud, using Amazon's Elastic Map Reduce service. > “That [EMR] got us to a place where we think about what data we collect. At the time it was really inefficient from a cost perspective, but there weren’t any creative constraints,” > The team eventually decided to bring it in house. “Our Hadoop cluster wields a huge amount of flexibility," said Rembetsy. "We can get a lot done on bare metal, and we know very much what we’re looking for. Cloud services, that’s where efficiencies fall away. Spin up and spin down doesn’t have the same value proposition.” [1] http://www.datacenterknowledge.com/archives/2015/05/11/how-etsy-optimized-data-center-infrastructure-for-efficiency http://www.datacenterknowledge.com/archives/2015/05/11/how-e...

interskh 9y ago
EMR is really expensive, especially you are using spot instances. It is roughly $2 for the EMR tax on $1 instance

josephhurtado 9y ago

It begs the question why Google Cloud and not MS Azure or Amazon's AWS? Could it be a price break they got, and also the AI advantages of Google Cloud's offerings? I am sure that played a big part.

oh-kumudo 9y ago
Probably because Google extends really really good deal to them for the PR effect. I don't think Etsy should consider itself as a competitor to Amazon, they simply can't. On the other side, AWS already has so many success stories that they are unlikely cater to Etsy specifically. That makes GCE the best option out there. As to Azure, I think people trust Google more as to cloud-based technology than Microsoft.
- bpicolo 9y ago
  > they simply can't Amazon is one of Etsy's most relevant competitors. They'd be insane not to consider them a competitor https://www.amazon.com/Handmade https://www.amazon.com/Handmade
bckygldstn 9y ago
Many retail companies avoid AWS because they see Amazon as a competitor. Source: work for a retail company, we went with Azure.
- TheReveller 9y ago
  I think that certainly comes into it. Seems perverse to take the money you make from sales and give a cut of it to your competitor, who can then reduce the prices in their own store.
  - billmalarky 9y ago
    That and there's a trust issue I'm sure, regarding hosting all your data with your competitor.
9y ago

code_duck 9y ago

Sounds good. I was an early Etsy seller, as a glass artist and bead vendor, As well as a developer who worked with them as closely as I could for a few years. They have always seemed a bit disconnected from their customers. The first regime, Rob Kalin, Haim and Revolving Dork, had an age and culture disconnect from most of their customers. The next group, headed by Chad Dickerson and Allspaw, seemed like they wanted to focus on technology for technology’s sake, more than what Etsy was actually doing with that as a company. They even made some statements to the effect of how they wanted Etsy to be known as a technology company, like Facebook, and be a platform, like Amazon. All fine and good, but so far, what all of the leaders of Etsy have had in common is that they don’t really understand what it’s like to be a micro business person selling handmade art online. If I was an investor, I wouldn’t want to be paying for a bunch of guys to experiment with Mongo DB search engines and shards. As long as they’re still not understanding their core business, it’s a distraction. Etsy should be focused on one thing: connecting buyers and sellers of handmade art with each other.

BlanketApple 9y ago
Disclaimer: I worked at Etsy, a long time ago. > The next group, headed by Chad Dickerson and Allspaw, seemed like they wanted to focus on technology for technology’s sake, more than what Etsy was actually doing with that as a company. I think you're missing quite a bit of this story. I was around during the transition of Chad from CTO -> CEO, and the truth is that Etsy needed to focus on the technology badly during that time. The site would go down constantly. We had very little insight into what was going on with sellers or buyers. Every busy day was basically us crossing our fingers that the site wouldn't go down. We needed someone focused heavily on the technology. We could never have continued on that path and grown as much as we did. > Etsy should be focused on one thing: connecting buyers and sellers of handmade art with each other. Deciding what potential buyers are interested in and what items to display to them is a pretty difficult technology problem. Detecting mass-produced goods in a way that's more subtle than "ban all Chinese sellers" is a pretty difficult technology problem. If anything I would argue that Etsy didn't focus nearly enough on that, and focused way too much on "low-hanging fruit" (minor workflow adjustments for sellers and buyers).

stareatgoats 9y ago

Not much meat for the technically interested here unfortunately. The respective business departments were clearly in charge of this press release. Which was a business mistake in my opinion: these press releases are primarily interesting for the CTOs and similar - the fluff presented here would serve to remind them of how they are kept at arms length i many board rooms even in their areas of expertise, more than anything. My guess is both Google Cloud and Etsy would dip in their opinion as result of it.

damm 9y ago

Okay so how much did they get for free by moving to Google Cloud? That's the question you should be asking. Etsy has been hurting; why would they move infrastructure now unless t hey got kind of kickback for the PR? This isn't an easy move; so there has to be a huge benefit to them.

CaptSpify 9y ago

Another DC to cloud story! I love these! It's like watching a play in 3 acts: Act 1) Yay, we did it, we're gonna save so much money! Act 2) [a year later] Wait, we're spending how much money?!? Act 3) It's going to cost us how much to leave our current cloud?!? Haven't people figured out yet that cloud companies make a profit? Sure, companies like Etsy can get a huge discount, but everyone else who isn't a big name is going to pay out the ass.

btian 9y ago
That's not how any of this works. Boeing makes a profit, but how much money will it cost to design and make a plane from scratch?
- CaptSpify 9y ago
  > That's not how any of this works. Weird then, considering that I've seen >5 companies go through that process. Something that you say doesn't work one way clearly does. Planes are vastly different than data-centers. Depending on what you need, a brand new data-center can run as little as a couple hundred dollars for a server + parts, and <$30 / month on everything else. What is the cheapest plane you can buy?

_Codemonkeyism 9y ago

I would love to see a cost analysis of a company moving to GCP/AWS. We were a 6M customer flash sale company sold for $200M. We had 3 devops engineers mostly helping developers with designing the system and pager duty - mainly because 24/7 was to demanding for only 2 people. We were SOX/PCI compliant. Hosting costs all in all by year were around the same as the salaries I'd say. People who moved to AWS/GCP, would I save costs in this scenario moving to AWS/GCP?