4 ms·

Your First Hire Should be a Sysadmin

36 points by knighthacker 13y ago

herge 13y ago
We are a small 5 programmer company, and we implemented a couple of ansible scripts to deploy our applications a 2-3 month project. For us, at least, a full-time sysadmin is overkill, and I would expect any decent programmer to be able to implement this himself as part of his basic skill set, or to be able to learn it quickly.
rstevenson542 13y ago
No... Just no.
knodi 13y ago
No, a full-time sysadmin is dumb idea for first hire. Your first deployment is not going to be complex and there will be a lot of moving parts. If you dev team can't understand deployment I'm not sure what your expecting your first hire sysadmin to do. Especially when services like heroku and take alot of the heavy lifting of the start ups hands, initially anyway.
jameszhang 13y ago
I really disagree with this. For your first hire, a full time sysadmin who cannot work on developing the product itself is just as useless as hiring a developer who cannot ship the code. I think what this article is trying to get at is that your fire hire should be a full-stack engineer who can also play the DevOps role when needed.
- throughnothing 13y ago
  I think saying a sysadmin 'cannot work on developing the product itself', is silly. I think it's a given that anyone joining at such an early stage of a company is going to be developing the product. You will definitely need a sysadmin that can develop the product and move fast. Your architecture is part of the product, and the sysadmin should help with the design and implementation of the architecture, as well as possibly parts of the codebase going onto that architecture.
charlesju 13y ago
I'm also going to have to disagree with this one. There are plenty of frameworks (Ruby on Rails) and companies (Engine Yard, Heroku) that take care of these issues for you on a time-share basis (ie. 20 companies use them, they hire 5 sysadmins). I would say if you're not spending 7 figures on server costs a year you absolutely do not need a personal sysadmin.
- druiid 13y ago
  Disagree with the seven figures number. Six figures, sure. Seven is just silly. You can have a lot of gear or servers for mid/low six figures.
kurtle 13y ago
Your first hire should be the person who fills in the missing gaps in your product. For example, good product, code is done, bad design => hire designer.
- throughnothing 13y ago
  yes!
j_baker 13y ago
I don't think you should hire a sysadmin until you have a larger set of developers. Still, I think there's a good point here: you should think about deployment early on. Having things automated will allow your early team to spend more time writing features and less time fighting fires and hacking production configurations.
johngalt 13y ago
Sysadmins are what you bring in once you've grown enough that operational issues detract from development rather than add to it. A sysadmin should absolutely not be your first hire. At most a consult with an engineer in the early days to get you started. This would help if your coders are used to handing off deployment to someone else. Bluntly, if you completely lack ops knowledge, buy an hour block from an ops specialist that can get you over the rough patches. My concern with a sysadmin first is that you'll want your early development very tight with your production systems. A sysadmin is the last thing you'll need while doing fast iterations on an MVP. In the early days your need for feedback is high and your operational issues are low. After some significant growth your ops needs will be high, and the feedback value of those issues will become lower (more repetitive less new information). This is where you bring in a sysadmin to manage/automate the repetitive and filter some of the ops issues into actionable information for the development team. Edit: Ops guy for over a decade btw.
- falcolas 13y ago
  > A sysadmin is the last thing you'll need while doing fast iterations on an MVP I'm not sure I agree with this. Someone needs to be working on your CM scripts and debugging production issues... and if it's not a SA, it's you or your developers. If you really want to iterate as quickly as possible, you need your developers focused on developing, not system administration.
  - johngalt 13y ago
    Sounds like you agree with everything I said until you edited my comment down to that sentence. If there's honestly 40 hours a week of ops issues during the first hire/MVP/market fitting phase. One of two things is happening: 1. Your startup is so wildly successful out of the gate that operations/infrastructure needs to grow exponentially. 2. Your startup has produced something that is complete garbage. Neither situation is likely, but I'll let you guess which scenario is more common. A much more realistic scenario is that ops work starts off as 5hours/week and grows to 10, 20, 30, 40 etc.. over a couple years without anyone noticing. By that time you've got a team of 4-6 devs and operations issues have become an annoying interruption to all of them. This is where bad IT decisions happen.
falcolas 13y ago
I imagine it depends on your focus. Want to pay someone else to do your sysadmin for you and take advantage of the economies of scale - use a PaaS provider and makes plans for when you outgrow it. Want your developers spending time debugging CM scripts and install quirks? "Full stack" developers it is. Want developers developing, and your application doesn't fit into a PaaS architecture? You absolutely need to hire a sysadmin.
wpeterson 13y ago
I could not disagree with this more. Focus on building your business and your product, not devops infrastructure. Unless your business is devops infrastructure. Your initial technology stack should allow you to deploy code seemlessly from day 1. This is why Heroku is a great start for early startups, despite the drawbacks.
- knighthacker 13y ago
  Having a DevOps first was THE reason we were able to focus on building the product and not invest time figuring out infrastructure.
- nasalgoat 13y ago
  Having spent nearly a decade of my two decade career cleaning up totally broken developer-architected systems that were so poorly coded for performance that they could barely handle 10 concurrent users, I have to strongly disagree. It's getting better, but most developers still focus on making it work right now over making it work right. They don't think about caching, or query optimization, or debug logging, or stateless scenarios for portability and clustering. These are all things that need to be thought about before you start. That said, a generic sysadmin won't be the kind of architect that is best, but they'll be thinking about more of that stuff than the devs, plus he can implement and manage your infrastructure. Seems like a win to me.
awnstudio 13y ago
Many of the comments seem to have relegated system administration to "code deployment" which is a gross understatement. Ansible, Puppet, and Chef are great configuration management tools but someone still has to do the configuring, and not configuring a system properly can be disastrous, especially for a startup dealing with financial information. Additionally, I feel that it's ridiculous and plain implausible to think that web application frameworks, e.g. Ruby-on-Rails, or a hosting companies, e.g. Engine Yard, Heroku, etc, should be responsible for your system configuration and network security.
CreakyParrot 13y ago
Why was the title changed from "Should" to "Could"? While I don't agree with the author's argument, he's at least making a specific point. "Could", on the other hand, reduces it to being meaningless. Your First Hire Could Be a Weirkeeper. Or a milliner. Or a janitor. Or, you know, anything.
- knighthacker 13y ago
  The post was changed by a moderator. Our blog post title is what we stand by
  - CreakyParrot 13y ago
    Sorry if I wasn't clear; that's what I assumed happened. Which seems an awfully silly thing for a moderator to do since it effectively guts the premise of the article. Glad to see it's been changed back.
goggles99 13y ago
>Your First Hire Should be a Sysadmin (an imperative and definitive statement) Written by a Sysadmin.... He goes on to say how he built infrastructure and how it benefited the company but... He was far from the first hire at Crowdtilt - which invalidates the whole premise. why compromise the whole integrity of the article with a false premise and untrue suggestion.
- mago0 13y ago
  The title is strong, yes (on purpose). I'm not certain how you came to the conclusion that I was far from the first hire at Crowdtilt. Your statement is simply false.

36 points by knighthacker 13y ago

herge 13y ago

We are a small 5 programmer company, and we implemented a couple of ansible scripts to deploy our applications a 2-3 month project. For us, at least, a full-time sysadmin is overkill, and I would expect any decent programmer to be able to implement this himself as part of his basic skill set, or to be able to learn it quickly.

rstevenson542 13y ago

No... Just no.

knodi 13y ago

No, a full-time sysadmin is dumb idea for first hire. Your first deployment is not going to be complex and there will be a lot of moving parts. If you dev team can't understand deployment I'm not sure what your expecting your first hire sysadmin to do. Especially when services like heroku and take alot of the heavy lifting of the start ups hands, initially anyway.

jameszhang 13y ago

I really disagree with this. For your first hire, a full time sysadmin who cannot work on developing the product itself is just as useless as hiring a developer who cannot ship the code. I think what this article is trying to get at is that your fire hire should be a full-stack engineer who can also play the DevOps role when needed.

throughnothing 13y ago
I think saying a sysadmin 'cannot work on developing the product itself', is silly. I think it's a given that anyone joining at such an early stage of a company is going to be developing the product. You will definitely need a sysadmin that can develop the product and move fast. Your architecture is part of the product, and the sysadmin should help with the design and implementation of the architecture, as well as possibly parts of the codebase going onto that architecture.

charlesju 13y ago

I'm also going to have to disagree with this one. There are plenty of frameworks (Ruby on Rails) and companies (Engine Yard, Heroku) that take care of these issues for you on a time-share basis (ie. 20 companies use them, they hire 5 sysadmins). I would say if you're not spending 7 figures on server costs a year you absolutely do not need a personal sysadmin.

druiid 13y ago
Disagree with the seven figures number. Six figures, sure. Seven is just silly. You can have a lot of gear or servers for mid/low six figures.

kurtle 13y ago

Your first hire should be the person who fills in the missing gaps in your product. For example, good product, code is done, bad design => hire designer.

throughnothing 13y ago
yes!

j_baker 13y ago

I don't think you should hire a sysadmin until you have a larger set of developers. Still, I think there's a good point here: you should think about deployment early on. Having things automated will allow your early team to spend more time writing features and less time fighting fires and hacking production configurations.

johngalt 13y ago

Sysadmins are what you bring in once you've grown enough that operational issues detract from development rather than add to it. A sysadmin should absolutely not be your first hire. At most a consult with an engineer in the early days to get you started. This would help if your coders are used to handing off deployment to someone else. Bluntly, if you completely lack ops knowledge, buy an hour block from an ops specialist that can get you over the rough patches. My concern with a sysadmin first is that you'll want your early development very tight with your production systems. A sysadmin is the last thing you'll need while doing fast iterations on an MVP. In the early days your need for feedback is high and your operational issues are low. After some significant growth your ops needs will be high, and the feedback value of those issues will become lower (more repetitive less new information). This is where you bring in a sysadmin to manage/automate the repetitive and filter some of the ops issues into actionable information for the development team. Edit: Ops guy for over a decade btw.

falcolas 13y ago
> A sysadmin is the last thing you'll need while doing fast iterations on an MVP I'm not sure I agree with this. Someone needs to be working on your CM scripts and debugging production issues... and if it's not a SA, it's you or your developers. If you really want to iterate as quickly as possible, you need your developers focused on developing, not system administration.
- johngalt 13y ago
  Sounds like you agree with everything I said until you edited my comment down to that sentence. If there's honestly 40 hours a week of ops issues during the first hire/MVP/market fitting phase. One of two things is happening: 1. Your startup is so wildly successful out of the gate that operations/infrastructure needs to grow exponentially. 2. Your startup has produced something that is complete garbage. Neither situation is likely, but I'll let you guess which scenario is more common. A much more realistic scenario is that ops work starts off as 5hours/week and grows to 10, 20, 30, 40 etc.. over a couple years without anyone noticing. By that time you've got a team of 4-6 devs and operations issues have become an annoying interruption to all of them. This is where bad IT decisions happen.

falcolas 13y ago

I imagine it depends on your focus. Want to pay someone else to do your sysadmin for you and take advantage of the economies of scale - use a PaaS provider and makes plans for when you outgrow it. Want your developers spending time debugging CM scripts and install quirks? "Full stack" developers it is. Want developers developing, and your application doesn't fit into a PaaS architecture? You absolutely need to hire a sysadmin.

wpeterson 13y ago

I could not disagree with this more. Focus on building your business and your product, not devops infrastructure. Unless your business is devops infrastructure. Your initial technology stack should allow you to deploy code seemlessly from day 1. This is why Heroku is a great start for early startups, despite the drawbacks.

knighthacker 13y ago
Having a DevOps first was THE reason we were able to focus on building the product and not invest time figuring out infrastructure.
nasalgoat 13y ago
Having spent nearly a decade of my two decade career cleaning up totally broken developer-architected systems that were so poorly coded for performance that they could barely handle 10 concurrent users, I have to strongly disagree. It's getting better, but most developers still focus on making it work right now over making it work right. They don't think about caching, or query optimization, or debug logging, or stateless scenarios for portability and clustering. These are all things that need to be thought about before you start. That said, a generic sysadmin won't be the kind of architect that is best, but they'll be thinking about more of that stuff than the devs, plus he can implement and manage your infrastructure. Seems like a win to me.

awnstudio 13y ago

Many of the comments seem to have relegated system administration to "code deployment" which is a gross understatement. Ansible, Puppet, and Chef are great configuration management tools but someone still has to do the configuring, and not configuring a system properly can be disastrous, especially for a startup dealing with financial information. Additionally, I feel that it's ridiculous and plain implausible to think that web application frameworks, e.g. Ruby-on-Rails, or a hosting companies, e.g. Engine Yard, Heroku, etc, should be responsible for your system configuration and network security.

CreakyParrot 13y ago

Why was the title changed from "Should" to "Could"? While I don't agree with the author's argument, he's at least making a specific point. "Could", on the other hand, reduces it to being meaningless. Your First Hire Could Be a Weirkeeper. Or a milliner. Or a janitor. Or, you know, anything.

knighthacker 13y ago
The post was changed by a moderator. Our blog post title is what we stand by
- CreakyParrot 13y ago
  Sorry if I wasn't clear; that's what I assumed happened. Which seems an awfully silly thing for a moderator to do since it effectively guts the premise of the article. Glad to see it's been changed back.

goggles99 13y ago

>Your First Hire Should be a Sysadmin (an imperative and definitive statement) Written by a Sysadmin.... He goes on to say how he built infrastructure and how it benefited the company but... He was far from the first hire at Crowdtilt - which invalidates the whole premise. why compromise the whole integrity of the article with a false premise and untrue suggestion.

mago0 13y ago
The title is strong, yes (on purpose). I'm not certain how you came to the conclusion that I was far from the first hire at Crowdtilt. Your statement is simply false.