7 ms·

Etsy CTO on Its Conservatively Crafty Tech Philosophy

62 points by amalantony06 11y ago

hanlec 11y ago
> One way Etsy drives this kind of commitment to deep understanding that seems to underlie its technology choices, is limiting the number of tools it uses as a company. It would rather be the master of few tools than have a broad knowledge across many different ones. Every large(-ish) company I know has pretty strict rules about the languages, frameworks, and tools used for the main products. The more flexible of these do allow prototyping using a different toolsets. The ones that have a developer audience always experiment with new environments. New tools and new processed get adopted over time, but in many cases the cycle they go through emulated pretty closely the adoption cycle seen outside. Even if the new shinny toy syndrome is very strong in this field, this makes a lot of sense, as nobody really wants to throw away everything. (_While writing this I got to think about the numerous rewrites eBay has seen over years_). > The company owns and operates its hardware and networks in its own datacenter. Part of the reason for avoiding the cloud is control, but not necessarily just from a security perspective as you might think. The real reason is Etsy personnel want to get under the hood and understand every aspect of every piece of technology the company controls. I continue to be very curious about how polarizing the cloud remains.
mpdehaan2 11y ago
> "The company has 685 employees of which approximately a third are engineers." Given that they've produced some impressive things, Etsy always seemed to be the example of a shop that was overdoing it - but I had actually assumed they were overcomplicating it with 20 engineers or so. Looking at their github now, and hearing those recent numbers, it feels much worse. That level of work is great, but for a company that does what Etsy does, much of it seems like overkill. (Managing 1000 instances should be the same as managing 100, this is where automation kicks in. Architecture enables that.) Ultimately, that's a tremendously large amount of engineers, to where I can't really mentally assign what all but about 20 of them might do. Let's be generous and say 30 - or even 40 :) All of that eats into the bottom line (and also makes adding more staff harder, as your tech becomes more and more custom). Really, at the basic level, it's a CRUD-based storefront website. Yes, it has scaling needs. But it's not really novel in that regard. Yet, they've built more automation around their automation (lots of things in the Chef ecosystem) and probably made things a lot more custom. I can understand some of this for say, multiplayer gaming or video streaming infrastructure maybe, but I would suspect Etsy could have been run a lot easier with less wheel reinvention. Startups should really strive, IMHO, to write as little of software as possible, that's outside of their line of business. When wheels are square, build better wheels sure, but what I see from that number looks like MASSIVE levels of maintaince costs for being that custom. So anyway, thanks for producing things that other people use and that are useful - YES - but from the startup accounting / efficiency perspective, I wouldn't want to see my company run that way.
- callahad 11y ago
  Not that they're necessarily in the same league, but would you say the same about Amazon?
  - mpdehaan2 11y ago
    Completely different domains and polar opposites on levels of technical requirement. Etsy is an end user web application (don't get me wrong, it's a great service). AWS is pretty much the pioneer in IaaS. I'd say Amazon (AWS) built the system so folks like Etsy wouldn't have to build as much as a system.
    - burkemw3 11y ago
      When I read callahad's comment, I thought it was referring to Amazon as a "CRUD-based storefront website". What would you say about the retail Amazon store compared to Etsy?
      - giaour 11y ago
        Well, Amazon also had to define what it meant to be a web store at scale. I think that mpdehaan2's point was that (from the outside looking in, at least) almost everything Etsy's doing seems to be a minor variation on a solved problem. That was definitely not the case for Amazon.
    - krschultz 11y ago
      That's getting the causality backwards. AWS exists because Amazon had so many engineers working on these problems it made sense to generalize it as a separate product. It wasn't like Amazon didn't have a lot of engineers but decided to build AWS so then hired a bunch of engineers.
    - stephenr 11y ago
      Well technically, Amazon built AWS so they could make money from other companies who can't, or don't want to, run their own infrastructure. At the scale of Etsy, they don't get any benefit from AWS - they're certainly big enough to justify the cost of an operations team, at which point AWS is just an expensive exercise in vendor-lockin. It really surprises me how many people think that being more self-reliant and understanding/controlling more of your dependencies, is a bad thing. It's not always possible, but why wouldn't you, if you had the resources to do so? The only time I got close to an answer on stuff like this was when I asked why the company behind a large tv/broadcast network insisted on using "vendors" for every little piece of technology - they would pay for hosted forums and battle through the issues with integration (of which there were many) rather than picking up an existing forum product and customising it as necessary. The only answer I was ever given: they want someone to sue when it goes to shit.
overgard 11y ago
My reaction to this as a professional programmer: Etsy sounds like a terrible place to work and the CTO is a moron.
- stephenr 11y ago
  Because they're "conservative" and don't change their entire stack to use the latest cool-kid technology every 6 months? Because they recognise that devops means something other than "we let our developers run wild in prod"?
  - fsk 11y ago
    At one place I worked, they let developers use whatever tools they wanted. The result was that their website was dependent on 50 Javascript packages that took forever to load. They had parts of their system written in Javascript, PHP, Ruby, Python, and .NET. That's how you get job ads that say "Candidate must have 2+ years experience in Python, PHP, Ruby, C#, and Javascript." Imagine putting a big effort into Angular 1.0, only to discover a year later that it's no longer going to be supported. Is it so bad for Etsy to be avoiding problems like that?
    - stephenr 11y ago
      Oh I believe it - I've worked on projects where a single developer adds a dozen JS libraries/jquery plugins/whatever because there is no concept of how maintainable it is long term. I don't even really like to call this "conservative". To me, "conservative" would be no XHR, no custom fonts, etc - the sort of stuff you know will work on IE5. I would consider the approach Etsy uses simply as "sane".
    - makeitsuckless 11y ago
      The problem in this scenario is not letting the developers use whatever they wanted. The problem is hiring the wrong developers, ones that use that freedom irresponsibly. Keeping developers on a short leash doesn't solve that problem, it just covers it up. And it becomes a self-fulfilling prophecy, because any developer capable of making their own decisions won't work there, so what you end up with is people that need that level of micromanagement. (Yes, this is a form of micromanagement.) And that's what I read in the whole article: an inexperienced CTO who has shaped his policy based on his own limited experience, not acknowledging that that vision is heavily influenced by his own weaknesses. He is projecting his own attitude as a developer on all developers, and extrapolates from there.
deleted 11y ago
[deleted]

62 points by amalantony06 11y ago

> One way Etsy drives this kind of commitment to deep understanding that seems to underlie its technology choices, is limiting the number of tools it uses as a company. It would rather be the master of few tools than have a broad knowledge across many different ones. Every large(-ish) company I know has pretty strict rules about the languages, frameworks, and tools used for the main products. The more flexible of these do allow prototyping using a different toolsets. The ones that have a developer audience always experiment with new environments. New tools and new processed get adopted over time, but in many cases the cycle they go through emulated pretty closely the adoption cycle seen outside. Even if the new shinny toy syndrome is very strong in this field, this makes a lot of sense, as nobody really wants to throw away everything. (_While writing this I got to think about the numerous rewrites eBay has seen over years_). > The company owns and operates its hardware and networks in its own datacenter. Part of the reason for avoiding the cloud is control, but not necessarily just from a security perspective as you might think. The real reason is Etsy personnel want to get under the hood and understand every aspect of every piece of technology the company controls. I continue to be very curious about how polarizing the cloud remains.

mpdehaan2 11y ago

> "The company has 685 employees of which approximately a third are engineers." Given that they've produced some impressive things, Etsy always seemed to be the example of a shop that was overdoing it - but I had actually assumed they were overcomplicating it with 20 engineers or so. Looking at their github now, and hearing those recent numbers, it feels much worse. That level of work is great, but for a company that does what Etsy does, much of it seems like overkill. (Managing 1000 instances should be the same as managing 100, this is where automation kicks in. Architecture enables that.) Ultimately, that's a tremendously large amount of engineers, to where I can't really mentally assign what all but about 20 of them might do. Let's be generous and say 30 - or even 40 :) All of that eats into the bottom line (and also makes adding more staff harder, as your tech becomes more and more custom). Really, at the basic level, it's a CRUD-based storefront website. Yes, it has scaling needs. But it's not really novel in that regard. Yet, they've built more automation around their automation (lots of things in the Chef ecosystem) and probably made things a lot more custom. I can understand some of this for say, multiplayer gaming or video streaming infrastructure maybe, but I would suspect Etsy could have been run a lot easier with less wheel reinvention. Startups should really strive, IMHO, to write as little of software as possible, that's outside of their line of business. When wheels are square, build better wheels sure, but what I see from that number looks like MASSIVE levels of maintaince costs for being that custom. So anyway, thanks for producing things that other people use and that are useful - YES - but from the startup accounting / efficiency perspective, I wouldn't want to see my company run that way.

callahad 11y ago
Not that they're necessarily in the same league, but would you say the same about Amazon?
- mpdehaan2 11y ago
  Completely different domains and polar opposites on levels of technical requirement. Etsy is an end user web application (don't get me wrong, it's a great service). AWS is pretty much the pioneer in IaaS. I'd say Amazon (AWS) built the system so folks like Etsy wouldn't have to build as much as a system.
  - burkemw3 11y ago
    When I read callahad's comment, I thought it was referring to Amazon as a "CRUD-based storefront website". What would you say about the retail Amazon store compared to Etsy?
    - giaour 11y ago
      Well, Amazon also had to define what it meant to be a web store at scale. I think that mpdehaan2's point was that (from the outside looking in, at least) almost everything Etsy's doing seems to be a minor variation on a solved problem. That was definitely not the case for Amazon.
  - krschultz 11y ago
    That's getting the causality backwards. AWS exists because Amazon had so many engineers working on these problems it made sense to generalize it as a separate product. It wasn't like Amazon didn't have a lot of engineers but decided to build AWS so then hired a bunch of engineers.
  - stephenr 11y ago
    Well technically, Amazon built AWS so they could make money from other companies who can't, or don't want to, run their own infrastructure. At the scale of Etsy, they don't get any benefit from AWS - they're certainly big enough to justify the cost of an operations team, at which point AWS is just an expensive exercise in vendor-lockin. It really surprises me how many people think that being more self-reliant and understanding/controlling more of your dependencies, is a bad thing. It's not always possible, but why wouldn't you, if you had the resources to do so? The only time I got close to an answer on stuff like this was when I asked why the company behind a large tv/broadcast network insisted on using "vendors" for every little piece of technology - they would pay for hosted forums and battle through the issues with integration (of which there were many) rather than picking up an existing forum product and customising it as necessary. The only answer I was ever given: they want someone to sue when it goes to shit.

overgard 11y ago

My reaction to this as a professional programmer: Etsy sounds like a terrible place to work and the CTO is a moron.

stephenr 11y ago
Because they're "conservative" and don't change their entire stack to use the latest cool-kid technology every 6 months? Because they recognise that devops means something other than "we let our developers run wild in prod"?
- fsk 11y ago
  At one place I worked, they let developers use whatever tools they wanted. The result was that their website was dependent on 50 Javascript packages that took forever to load. They had parts of their system written in Javascript, PHP, Ruby, Python, and .NET. That's how you get job ads that say "Candidate must have 2+ years experience in Python, PHP, Ruby, C#, and Javascript." Imagine putting a big effort into Angular 1.0, only to discover a year later that it's no longer going to be supported. Is it so bad for Etsy to be avoiding problems like that?
  - stephenr 11y ago
    Oh I believe it - I've worked on projects where a single developer adds a dozen JS libraries/jquery plugins/whatever because there is no concept of how maintainable it is long term. I don't even really like to call this "conservative". To me, "conservative" would be no XHR, no custom fonts, etc - the sort of stuff you know will work on IE5. I would consider the approach Etsy uses simply as "sane".
  - makeitsuckless 11y ago
    The problem in this scenario is not letting the developers use whatever they wanted. The problem is hiring the wrong developers, ones that use that freedom irresponsibly. Keeping developers on a short leash doesn't solve that problem, it just covers it up. And it becomes a self-fulfilling prophecy, because any developer capable of making their own decisions won't work there, so what you end up with is people that need that level of micromanagement. (Yes, this is a form of micromanagement.) And that's what I read in the whole article: an inexperienced CTO who has shaped his policy based on his own limited experience, not acknowledging that that vision is heavily influenced by his own weaknesses. He is projecting his own attitude as a developer on all developers, and extrapolates from there.

deleted 11y ago

[deleted]