4 ms·

I agree - and it's not just what gets you promoted, but also what gets you hired, and what people look for in general. You're looking for your first DevOps per

I agree - and it's not just what gets you promoted, but also what gets you hired, and what people look for in general. You're looking for your first DevOps person, so you want someone who has experience doing DevOps. They'll tell you about all the fancy frameworks and tooling they've used to do Serious Business™, and you'll be impressed and hire them. They'll then proceed to do exactly that for your company, and you'll feel good because you feel it sets you up for the future. Nobody's against it. So you end up in that situation, which even a basic home desktop would be more than capable of handling.

jrjeksjd8d 9mo ago
I have been the first (and only) DevOps person at a couple startups. I'm usually pretty guilty of NIH and wanting to develop in-house tooling to improve productivity. But more and more in my career I try to make boring choices. Cost is usually not a huge problem beyond seed stage. Series A-B the biggest problem is growing the customer base so the fixed infra costs become a rounding error. We've built the product and we're usually focused on customer enablement and technical wins - proving that the product works 100% of the time to large enterprises so we can close deals. We can't afford weird flakiness in the middle of a POC. Another factor I rarely see discussed is bus factor. I've been in the industry for over a decade, and I like to be able to go on vacation. It's nice to hand off the pager sometimes. Using established technologies makes it possible to delegate responsibility to the rest of the team, instead of me owning a little rats nest fiefdom of my own design. The fact is that if 5k/month infra cost for a core part of the service sinks your VC backed startup, you've got bigger problems. Investors gave you a big pile of money to go and get customers _now_. An extra month of runway isn't going to save you.
- woooooo 9mo ago
  The issue is when all the spending gets you is more complexity, maintenance, and you don't even get a performance benefit. I once interviewed with a company that did some machine learning stuff, this was a while back when that typically meant "1 layer of weights from a regression we run overnight every night". The company asked how I had solved the complex problem of getting the weights to inference servers. I said we had a 30 line shell script that ssh'd them over and then mv'd them into place. Meanwhile the application reopened the file every so often. Zero problems with it ever. They thought I was a caveman.
  - ffsm8 9mo ago
    The issue with solutions like that is usually that people don't know how it works and how to find it if it ever stops working... Basically discoverability is where shell script fail
    - chuckadams 9mo ago
      Those scripts have logs, right? Log a hostname and path when they run. If no one thinks to look at logs, then there's a bigger problem going on than a one-off script.
    - woooooo 9mo ago
      That becomes a problem if you let the shell script mutate into an "everything" script that's solving tons of business problems. Or if you're reinventing kubernetes with shell scripts. There's still a place for simple solutions to simple problems.
      - esafak 9mo ago
        That's happens naturally as every engineer adds just another feature to it.
    - justsomehnguy 9mo ago
      > Basically discoverability is where shell script fail No, it's lack of documentation and no amount of $$$$/m enterprise AI solutions (R)(TM) would help you if there is no documentation.
    - LevGoldstein 9mo ago
      Which is why you take the time to put usage docs in the repo README, make sure the script is packaged and deployed via the same methods that the rest of the company uses, and ensure that it logs success/failure conditions. That's been pretty standard at every organization I've been at my entire professional career. Anyone who can't manage that is going to create worse problems when designing/building/maintaining a more complex system.
- SJC_Hacker 9mo ago
  In my experience, that $5k/month easily blows up into $100k/month
pragma_x 9mo ago
I've seen the ramifications of this "CV first" kind of engineering. Let's just say that it's a bad time when you're saddled with tech debt solely from a handful of influential people that really just wanted to work elsewhere.
- bandrami 9mo ago
  I'm largely a stranger to the js world but from the outside it sure looks like projects are sharded so as to maximize npm contribution count
atomicnumber3 9mo ago
I'm convinced k8s is a conspiracy by bigtech to suppress startups.
- hunterpayne 9mo ago
  So its the EJBs of this age then?
exac 9mo ago
This. It is resume-driven development. Especially at startups where the engineers aren't compensated well enough or don't believe the produce can succeed.

jrjeksjd8d 9mo ago

I have been the first (and only) DevOps person at a couple startups. I'm usually pretty guilty of NIH and wanting to develop in-house tooling to improve productivity. But more and more in my career I try to make boring choices. Cost is usually not a huge problem beyond seed stage. Series A-B the biggest problem is growing the customer base so the fixed infra costs become a rounding error. We've built the product and we're usually focused on customer enablement and technical wins - proving that the product works 100% of the time to large enterprises so we can close deals. We can't afford weird flakiness in the middle of a POC. Another factor I rarely see discussed is bus factor. I've been in the industry for over a decade, and I like to be able to go on vacation. It's nice to hand off the pager sometimes. Using established technologies makes it possible to delegate responsibility to the rest of the team, instead of me owning a little rats nest fiefdom of my own design. The fact is that if 5k/month infra cost for a core part of the service sinks your VC backed startup, you've got bigger problems. Investors gave you a big pile of money to go and get customers _now_. An extra month of runway isn't going to save you.

woooooo 9mo ago
The issue is when all the spending gets you is more complexity, maintenance, and you don't even get a performance benefit. I once interviewed with a company that did some machine learning stuff, this was a while back when that typically meant "1 layer of weights from a regression we run overnight every night". The company asked how I had solved the complex problem of getting the weights to inference servers. I said we had a 30 line shell script that ssh'd them over and then mv'd them into place. Meanwhile the application reopened the file every so often. Zero problems with it ever. They thought I was a caveman.
- ffsm8 9mo ago
  The issue with solutions like that is usually that people don't know how it works and how to find it if it ever stops working... Basically discoverability is where shell script fail
  - chuckadams 9mo ago
    Those scripts have logs, right? Log a hostname and path when they run. If no one thinks to look at logs, then there's a bigger problem going on than a one-off script.
  - woooooo 9mo ago
    That becomes a problem if you let the shell script mutate into an "everything" script that's solving tons of business problems. Or if you're reinventing kubernetes with shell scripts. There's still a place for simple solutions to simple problems.
    - esafak 9mo ago
      That's happens naturally as every engineer adds just another feature to it.
  - justsomehnguy 9mo ago
    > Basically discoverability is where shell script fail No, it's lack of documentation and no amount of $$$$/m enterprise AI solutions (R)(TM) would help you if there is no documentation.
  - LevGoldstein 9mo ago
    Which is why you take the time to put usage docs in the repo README, make sure the script is packaged and deployed via the same methods that the rest of the company uses, and ensure that it logs success/failure conditions. That's been pretty standard at every organization I've been at my entire professional career. Anyone who can't manage that is going to create worse problems when designing/building/maintaining a more complex system.
SJC_Hacker 9mo ago
In my experience, that $5k/month easily blows up into $100k/month

pragma_x 9mo ago

I've seen the ramifications of this "CV first" kind of engineering. Let's just say that it's a bad time when you're saddled with tech debt solely from a handful of influential people that really just wanted to work elsewhere.

bandrami 9mo ago
I'm largely a stranger to the js world but from the outside it sure looks like projects are sharded so as to maximize npm contribution count

atomicnumber3 9mo ago

I'm convinced k8s is a conspiracy by bigtech to suppress startups.

hunterpayne 9mo ago
So its the EJBs of this age then?

exac 9mo ago

This. It is resume-driven development. Especially at startups where the engineers aren't compensated well enough or don't believe the produce can succeed.