6 ms·

> "The company has 685 employees of which approximately a third are engineers." Given that they've produced some impressive things, Etsy always seemed to be th

> "The company has 685 employees of which approximately a third are engineers." Given that they've produced some impressive things, Etsy always seemed to be the example of a shop that was overdoing it - but I had actually assumed they were overcomplicating it with 20 engineers or so. Looking at their github now, and hearing those recent numbers, it feels much worse. That level of work is great, but for a company that does what Etsy does, much of it seems like overkill. (Managing 1000 instances should be the same as managing 100, this is where automation kicks in. Architecture enables that.) Ultimately, that's a tremendously large amount of engineers, to where I can't really mentally assign what all but about 20 of them might do. Let's be generous and say 30 - or even 40 :) All of that eats into the bottom line (and also makes adding more staff harder, as your tech becomes more and more custom). Really, at the basic level, it's a CRUD-based storefront website. Yes, it has scaling needs. But it's not really novel in that regard. Yet, they've built more automation around their automation (lots of things in the Chef ecosystem) and probably made things a lot more custom. I can understand some of this for say, multiplayer gaming or video streaming infrastructure maybe, but I would suspect Etsy could have been run a lot easier with less wheel reinvention. Startups should really strive, IMHO, to write as little of software as possible, that's outside of their line of business. When wheels are square, build better wheels sure, but what I see from that number looks like MASSIVE levels of maintaince costs for being that custom. So anyway, thanks for producing things that other people use and that are useful - YES - but from the startup accounting / efficiency perspective, I wouldn't want to see my company run that way.

callahad 11y ago
Not that they're necessarily in the same league, but would you say the same about Amazon?
- mpdehaan2 11y ago
  Completely different domains and polar opposites on levels of technical requirement. Etsy is an end user web application (don't get me wrong, it's a great service). AWS is pretty much the pioneer in IaaS. I'd say Amazon (AWS) built the system so folks like Etsy wouldn't have to build as much as a system.
  - burkemw3 11y ago
    When I read callahad's comment, I thought it was referring to Amazon as a "CRUD-based storefront website". What would you say about the retail Amazon store compared to Etsy?
    - giaour 11y ago
      Well, Amazon also had to define what it meant to be a web store at scale. I think that mpdehaan2's point was that (from the outside looking in, at least) almost everything Etsy's doing seems to be a minor variation on a solved problem. That was definitely not the case for Amazon.
  - krschultz 11y ago
    That's getting the causality backwards. AWS exists because Amazon had so many engineers working on these problems it made sense to generalize it as a separate product. It wasn't like Amazon didn't have a lot of engineers but decided to build AWS so then hired a bunch of engineers.
  - stephenr 11y ago
    Well technically, Amazon built AWS so they could make money from other companies who can't, or don't want to, run their own infrastructure. At the scale of Etsy, they don't get any benefit from AWS - they're certainly big enough to justify the cost of an operations team, at which point AWS is just an expensive exercise in vendor-lockin. It really surprises me how many people think that being more self-reliant and understanding/controlling more of your dependencies, is a bad thing. It's not always possible, but why wouldn't you, if you had the resources to do so? The only time I got close to an answer on stuff like this was when I asked why the company behind a large tv/broadcast network insisted on using "vendors" for every little piece of technology - they would pay for hosted forums and battle through the issues with integration (of which there were many) rather than picking up an existing forum product and customising it as necessary. The only answer I was ever given: they want someone to sue when it goes to shit.
    - giaour 11y ago
- deleted 11y ago
  [deleted]
eikenberry 11y ago
Sounds like they really drank the commodity developer Kool-Aid. What I mean by this is heavy process/standards with tightly controlled tech stacks which attracts salaryman types. They don't try to attract talent, they just want enough bodies that enough work gets done to achieve their goals. It eliminates the need for recruitment by throwing money at the problem. I've mostly seen this at very larger companies but it is not restricted to them.
- coldcode 11y ago
  Programmers as assembly line workers. No thank you.
  - stephenr 11y ago
    Sounds to me more like programmers as programmers instead of programmers as rock star ninja cowboys
    - eikenberry 11y ago
      There are more types of people in the field than the salaryman going through the motions and the douchebag rock star stereotype.
Eridrus 11y ago
I think this article addresses what they do; they work their way down the stack and broaden what they provide to the business vs outsourcing to some SaaS company. Sure, you could run this in the cloud and not worry about physical machines, but that will probably be more expensive. As companies grow, it becomes cheaper to bring lots of things in house, rather than outsourcing them. As companies grow, small % differences can result in large total impact, at which point having engineers eek out small gains may make a lot of sense, e.g. think of Google's results around latency, or small improvements in recommendation engines that drive higher sales.
- jroseattle 11y ago
  > you could run this in the cloud and not worry about physical machines, but that will probably be more expensive. I question what the current cost structure looks like. As a public company, my guess is that investors will do the same as well. In a world where Instagram deals with orders of magnitude more customers with a whopping five engineers, it's a pretty high bar to show why so many resources are necessary for this particular operation. I'm sure the Etsy guys are sensitive to that as well, but from the outside it's hard to believe there isn't a better way to use resources more efficiently.
  - mcosta 11y ago
    I instagram loses a photo, nobody cares. If etsy loses an order is deal-breaker. Not the same problem domain. edit: Instagam has not that many customers. Has many users. That's a real diference.
    - jroseattle 11y ago
      Very good point. A transaction involving a user's money carries a great deal more weight in terms of reliability, accuracy, etc. That said, the resource difference between these two companies is considerably large, and the tech in question (e-commerce) is not exactly an area of unknown innovation. From a technical standpoint, that's sufficiently a solved problem.
- 11y ago
jasim 11y ago
Appreciate your insight on automation, and fully agree with the need for startups to use commodity software as much as possible and focus on business needs. However, I suspect most of the 200+ engineers would be needed to keep the business running in either case. They've written StatsD (Node), a custom deployment framework, an A/B testing framework and a bunch of other technology software (https://github.com/etsy https://github.com/etsy) that could need dedicated engineers to build and maintain. But all of this would pale compared to the needs of the business itself. At sufficiently large scale, any CRUD-based storefront website becomes an entire supply chain management system. I suspect there are many ways to slice up an SCM, but from a functional level, you'll need subsystems for each part of the domain: managing customer orders, warehouses, logistics, payments, offers & promotions, developer facing APIs, accounting & book-keeping, .. These systems can be built using a service-oriented architecture, but Etsy is famous for promoting the large monolithic app architecture (http://highscalability.com/blog/2014/7/28/the-great-microservices-vs-monolithic-apps-twitter-melee.html http://highscalability.com/blog/2014/7/28/the-great-microser...), which would remove the need for a lot of SOA plumbing work and communication overhead. Apart from the core business logic, they'll also need a whole bunch of QAs, Android and iOS developers, designers, DevOps folks, database admins, security specialists and so on. All told, they could probably run a leaner ship than the 200-odd engineers they now have, but I wonder how much more leaner.
- mpdehaan2 11y ago
  My main point was that writing all the custom stuff was why the staffing numbers were so high. One small correction - In this case they also do not have warehouses since items are shipped from craftspeople. I also would expect mobile apps could be done by a team of a handful of people. As for SOA vs monolithic apps (highscalability link currently is a 404), it's largely horizontal scaling and a load balancer. That in itself shouldn't change architectual choices.

callahad 11y ago

Not that they're necessarily in the same league, but would you say the same about Amazon?

mpdehaan2 11y ago
Completely different domains and polar opposites on levels of technical requirement. Etsy is an end user web application (don't get me wrong, it's a great service). AWS is pretty much the pioneer in IaaS. I'd say Amazon (AWS) built the system so folks like Etsy wouldn't have to build as much as a system.
- burkemw3 11y ago
  When I read callahad's comment, I thought it was referring to Amazon as a "CRUD-based storefront website". What would you say about the retail Amazon store compared to Etsy?
  - giaour 11y ago
    Well, Amazon also had to define what it meant to be a web store at scale. I think that mpdehaan2's point was that (from the outside looking in, at least) almost everything Etsy's doing seems to be a minor variation on a solved problem. That was definitely not the case for Amazon.
- krschultz 11y ago
  That's getting the causality backwards. AWS exists because Amazon had so many engineers working on these problems it made sense to generalize it as a separate product. It wasn't like Amazon didn't have a lot of engineers but decided to build AWS so then hired a bunch of engineers.
- stephenr 11y ago
  Well technically, Amazon built AWS so they could make money from other companies who can't, or don't want to, run their own infrastructure. At the scale of Etsy, they don't get any benefit from AWS - they're certainly big enough to justify the cost of an operations team, at which point AWS is just an expensive exercise in vendor-lockin. It really surprises me how many people think that being more self-reliant and understanding/controlling more of your dependencies, is a bad thing. It's not always possible, but why wouldn't you, if you had the resources to do so? The only time I got close to an answer on stuff like this was when I asked why the company behind a large tv/broadcast network insisted on using "vendors" for every little piece of technology - they would pay for hosted forums and battle through the issues with integration (of which there were many) rather than picking up an existing forum product and customising it as necessary. The only answer I was ever given: they want someone to sue when it goes to shit.
  - giaour 11y ago
deleted 11y ago
[deleted]

eikenberry 11y ago

Sounds like they really drank the commodity developer Kool-Aid. What I mean by this is heavy process/standards with tightly controlled tech stacks which attracts salaryman types. They don't try to attract talent, they just want enough bodies that enough work gets done to achieve their goals. It eliminates the need for recruitment by throwing money at the problem. I've mostly seen this at very larger companies but it is not restricted to them.

coldcode 11y ago
Programmers as assembly line workers. No thank you.
- stephenr 11y ago
  Sounds to me more like programmers as programmers instead of programmers as rock star ninja cowboys
  - eikenberry 11y ago
    There are more types of people in the field than the salaryman going through the motions and the douchebag rock star stereotype.

Eridrus 11y ago

I think this article addresses what they do; they work their way down the stack and broaden what they provide to the business vs outsourcing to some SaaS company. Sure, you could run this in the cloud and not worry about physical machines, but that will probably be more expensive. As companies grow, it becomes cheaper to bring lots of things in house, rather than outsourcing them. As companies grow, small % differences can result in large total impact, at which point having engineers eek out small gains may make a lot of sense, e.g. think of Google's results around latency, or small improvements in recommendation engines that drive higher sales.

jroseattle 11y ago
> you could run this in the cloud and not worry about physical machines, but that will probably be more expensive. I question what the current cost structure looks like. As a public company, my guess is that investors will do the same as well. In a world where Instagram deals with orders of magnitude more customers with a whopping five engineers, it's a pretty high bar to show why so many resources are necessary for this particular operation. I'm sure the Etsy guys are sensitive to that as well, but from the outside it's hard to believe there isn't a better way to use resources more efficiently.
- mcosta 11y ago
  I instagram loses a photo, nobody cares. If etsy loses an order is deal-breaker. Not the same problem domain. edit: Instagam has not that many customers. Has many users. That's a real diference.
  - jroseattle 11y ago
    Very good point. A transaction involving a user's money carries a great deal more weight in terms of reliability, accuracy, etc. That said, the resource difference between these two companies is considerably large, and the tech in question (e-commerce) is not exactly an area of unknown innovation. From a technical standpoint, that's sufficiently a solved problem.
11y ago

jasim 11y ago

Appreciate your insight on automation, and fully agree with the need for startups to use commodity software as much as possible and focus on business needs. However, I suspect most of the 200+ engineers would be needed to keep the business running in either case. They've written StatsD (Node), a custom deployment framework, an A/B testing framework and a bunch of other technology software (https://github.com/etsy https://github.com/etsy) that could need dedicated engineers to build and maintain. But all of this would pale compared to the needs of the business itself. At sufficiently large scale, any CRUD-based storefront website becomes an entire supply chain management system. I suspect there are many ways to slice up an SCM, but from a functional level, you'll need subsystems for each part of the domain: managing customer orders, warehouses, logistics, payments, offers & promotions, developer facing APIs, accounting & book-keeping, .. These systems can be built using a service-oriented architecture, but Etsy is famous for promoting the large monolithic app architecture (http://highscalability.com/blog/2014/7/28/the-great-microservices-vs-monolithic-apps-twitter-melee.html http://highscalability.com/blog/2014/7/28/the-great-microser...), which would remove the need for a lot of SOA plumbing work and communication overhead. Apart from the core business logic, they'll also need a whole bunch of QAs, Android and iOS developers, designers, DevOps folks, database admins, security specialists and so on. All told, they could probably run a leaner ship than the 200-odd engineers they now have, but I wonder how much more leaner.

mpdehaan2 11y ago
My main point was that writing all the custom stuff was why the staffing numbers were so high. One small correction - In this case they also do not have warehouses since items are shipped from craftspeople. I also would expect mobile apps could be done by a team of a handful of people. As for SOA vs monolithic apps (highscalability link currently is a 404), it's largely horizontal scaling and a load balancer. That in itself shouldn't change architectual choices.