6 ms·

Not a solo founder, but I do a lot of ops professionally. Unless you have a specific reason, I say it’s best to avoid complicated tooling until later. k8s, ci,

Not a solo founder, but I do a lot of ops professionally. Unless you have a specific reason, I say it’s best to avoid complicated tooling until later. k8s, ci, etc are all really useful and solve a lot of hard problems, but god are they a bitch to set up and then monitor, fix when something weird happens, secure from nasty people outside, patch when some update becomes necessary, resist the urge to “improve”, etc. This doesn’t include the time it takes to work these tools into your project, like k8s for example really requires you grok its model of how stuff it runs is supposed to look. There’s a reason entire companies exist that simply offer managed versions of these services! If you’re using fewer than like 10 boxes and don’t work with many people, you can get away with spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script, service or cron to keep running) and honestly it’s pretty fun to just run a command and poof you’re deployed. Then you can just wait until it starts making sense to use fancier tooling (more load, more sites/DCs, more services, more people/teams, etc). Monitoring is always good to start with, for other stuff a good litmus test is that price points for hosted k8s & ops stuff will start to make sense, you’ll be paying in time in any case. Who knows, maybe when that happens something new will pop up that makes even more sense. Until then, reducing the amount of stuff you need to worry about is always a good strategy.

noir_lord 5y ago
Agree with all of this. If you want an intermediate step you can just make docker images of your application and deploy and run those manually (again with shell scripts). You can get a long way that way without having to deal with kubernetes.
dbbk 5y ago
Or just deploy it on Heroku and call it a day?
- clairity 5y ago
  yah, this (or something similar) is the most straightforward solution. the whole point of heroku is to abstract away devops for solo/small dev groups and make deployments “one-click”. it’s unclear why that option was explicitly ruled out here.
oauea 5y ago
> If you’re using fewer than like 10 boxes and don’t work with many people, you can get away with spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script, service or cron to keep running) If you have multiple people on the team that have access to servers, this will shoot you in the foot a year or two later. Good luck replicating that server setup when no one has any clue whatsoever about all the small tweaks that were performed ad-hoc over the years.
- martijnvds 5y ago
  Next step up is something like Ansible to manage it all.
  - oauea 5y ago
    Yes. Then you realize that people may still SSH into your servers and manually tweak things when they need the fix "now". So then you start restricting SSH access and forcing everything through Ansible. And now you're deep in deployment best practices again ;)
    - malwrar 5y ago
      I think you’re describing the growing pains that justify paying the costs of more complicated tooling. The big difference is that instead of needing to figure it out as a solo founder with bigger problems, in this scenario you get to figure it out as a solo founder with a team and presumably some extra $$.
- effie 5y ago
  > Good luck replicating that server setup It is easy when the server is regularly backuped.
davedx 5y ago
> spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script I never realised I was using spooky arcane oldhat stuff! I feel wizardly now. My projects (for small clients and myself) basically use this. - A "build.sh" script that does a local build of back end and front end - A "deploy.sh" script that scp's everything to the server (either a digital ocean VPS or an EC2 instance), runs npm install, runs database migrations and restarts with pm2 So running my entire CI pipeline in terminal is: ./build.sh && ./deploy.sh
- leetrout 5y ago
  This is the way
- noduerme 5y ago
  Basically same here. In cases the deployment has to be zipped and manually deployed to elastic beanstalk. On some EC2 instances for projects running a PHP stack, I don't even bother with this, just SFTP script changes to the server straight out of my IDE. Take care of any rollback needs with local Git versioning.
- bootwoot 5y ago
  This is great. Until you need to roll back.
  - brightball 5y ago
    Keep a simple tool like Ansible around for those spooky admin tricks and you can take advantage of the Ansistrano plugins for smooth deploy and rollback (Ruby's great Capistrano tool ported to Ansible). https://ansistrano.com/ https://ansistrano.com/ It's pretty fantastic.
    - 1MachineElf 5y ago
      This looks awesome. I'm currently in the middle of learning Ansible now for my FjeeBSD jails. Have any other plugins to recommend?
nanis 5y ago
Indeed, for a bunch of hobby things, I use rsync -rvzC --no-p --no-g --chmod=ugo=rwX --stats -e "ssh . ${remote}:${path}" where `${remote}` is defined in `.ssh/config` and `${path}` is in the project config to move files over to a new deployment directory. Then, a quick command over SSH changes the symlink to the site's configuration file and restarts the web server. The sites are either on Linode or SSDNodes. Has been working for me for decades and don't need to change it for these hobby things. After all, the log files prove that these things get more traffic from exploitation probes than real people ;-)
- NortySpock 5y ago
  Also a hobbyist sysadmin at home, but after reading a few comments on the internet, I found that the rsync -z compress flag was bottlenecked on a maxed-out CPU thread of my Raspberry Pi 4 NAS transfers. Admittedly, these are mostly "local" drive-to-drive transfers over USB rather than server-to-server transfers over house-wide gigabit. But consider trying the transfer without compression. I'll definitely have to read up on your other flags though, if your rsync works from just one machine rather than both that might solve a minor problem case I have where the destination machine lacks rsync... https://unix.stackexchange.com/questions/188737/does-compression-option-z-with-rsync-speed-up-backup https://unix.stackexchange.com/questions/188737/does-compres...
  - nanis 5y ago
    Really good observation. All the settings are from a long time ago when link quality was way worse than what we get these days. I should review those options, definitely. > -C, --cvs-exclude auto-ignore files in the same way CVS does Just goes to show you how long I've been using this stuff :-) The additional options may not be relevant now that I am just deploying creating a new target directory each time.
marstall 5y ago
I'm fully on team spooky admin - however on my latest project, I've been loving being on the free tier of tools like vercel, which deploys from git commit, and firebase, which is mostly maintenance free. That really reduces the amount of devops in my day.
JTbane 5y ago
>you can get away with spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script, service or cron to keep running) You would be surprised at how many old enterprises still do this. Sure, it's ugly, but it can be simpler than k8s, and anyone with scripting chops can understand it.
wefarrell 5y ago
At this point CI is pretty easy to setup and monitor and it's integrated with practically every development/deployment tool, so I have to disagree about waiting to implement it.
- cutemonster 5y ago
  What do you use for CI?
NotAnOtter 5y ago
Yeesh I cannot imagine not setting up at least a basic CI. Even if you remove all unit/integration test alarms, a CI is still very useful. The 20 minutes of build -> upload -> deploy script -> check prod vs a 1/2 day to set up a CI and forget it. I'm not sure you save much time but you save a lot of headache
- emidln 5y ago
  I think you mean automation is very useful. I've worked in a lot of places where we had a single `make deploy` that could compile, test, upload the artifacts, deploy the artifacts to hosts, start the new services, and slowly transition the load balancer to point at the new instances while automatically backing out if error metrics started lighting up. This is just automation, it doesn't have to happen on every commit. Especially when working solo or on a small team, I might not want to maintain a lot of VCS branches and instead, just deploy when I feel it's ready. You have to do all of that stuff anyway, so if you later decide to hook it up to a VCS trigger, you're all set.
  - NotAnOtter 5y ago
    I guess that's fair, I'm probably coming into this having only seen it done a certain way and blind to other possibilities. That being said, it would take a lot to convince me a standard CI/CD pipeline is not worth the investment. My only experience with non CI/CD deployment process was a very manual process at a startup, then when I saw my first CI/CD it felt like magic.
    - smackeyacky 5y ago
      A good old makefile is a CI/CD pipeline. All the new fangled tools are re-inventing a lot of capabilities make already has. I blame the new kids on the block who have only ever known Windows :-)

noir_lord 5y ago

Agree with all of this. If you want an intermediate step you can just make docker images of your application and deploy and run those manually (again with shell scripts). You can get a long way that way without having to deal with kubernetes.

dbbk 5y ago

Or just deploy it on Heroku and call it a day?

clairity 5y ago
yah, this (or something similar) is the most straightforward solution. the whole point of heroku is to abstract away devops for solo/small dev groups and make deployments “one-click”. it’s unclear why that option was explicitly ruled out here.

oauea 5y ago

> If you’re using fewer than like 10 boxes and don’t work with many people, you can get away with spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script, service or cron to keep running) If you have multiple people on the team that have access to servers, this will shoot you in the foot a year or two later. Good luck replicating that server setup when no one has any clue whatsoever about all the small tweaks that were performed ad-hoc over the years.

martijnvds 5y ago
Next step up is something like Ansible to manage it all.
- oauea 5y ago
  Yes. Then you realize that people may still SSH into your servers and manually tweak things when they need the fix "now". So then you start restricting SSH access and forcing everything through Ansible. And now you're deep in deployment best practices again ;)
  - malwrar 5y ago
    I think you’re describing the growing pains that justify paying the costs of more complicated tooling. The big difference is that instead of needing to figure it out as a solo founder with bigger problems, in this scenario you get to figure it out as a solo founder with a team and presumably some extra $$.
effie 5y ago
> Good luck replicating that server setup It is easy when the server is regularly backuped.

davedx 5y ago

> spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script I never realised I was using spooky arcane oldhat stuff! I feel wizardly now. My projects (for small clients and myself) basically use this. - A "build.sh" script that does a local build of back end and front end - A "deploy.sh" script that scp's everything to the server (either a digital ocean VPS or an EC2 instance), runs npm install, runs database migrations and restarts with pm2 So running my entire CI pipeline in terminal is: ./build.sh && ./deploy.sh

leetrout 5y ago
This is the way
noduerme 5y ago
Basically same here. In cases the deployment has to be zipped and manually deployed to elastic beanstalk. On some EC2 instances for projects running a PHP stack, I don't even bother with this, just SFTP script changes to the server straight out of my IDE. Take care of any rollback needs with local Git versioning.
bootwoot 5y ago
This is great. Until you need to roll back.
- brightball 5y ago
  Keep a simple tool like Ansible around for those spooky admin tricks and you can take advantage of the Ansistrano plugins for smooth deploy and rollback (Ruby's great Capistrano tool ported to Ansible). https://ansistrano.com/ https://ansistrano.com/ It's pretty fantastic.
  - 1MachineElf 5y ago
    This looks awesome. I'm currently in the middle of learning Ansible now for my FjeeBSD jails. Have any other plugins to recommend?

nanis 5y ago

Indeed, for a bunch of hobby things, I use rsync -rvzC --no-p --no-g --chmod=ugo=rwX --stats -e "ssh . ${remote}:${path}" where `${remote}` is defined in `.ssh/config` and `${path}` is in the project config to move files over to a new deployment directory. Then, a quick command over SSH changes the symlink to the site's configuration file and restarts the web server. The sites are either on Linode or SSDNodes. Has been working for me for decades and don't need to change it for these hobby things. After all, the log files prove that these things get more traffic from exploitation probes than real people ;-)

NortySpock 5y ago
Also a hobbyist sysadmin at home, but after reading a few comments on the internet, I found that the rsync -z compress flag was bottlenecked on a maxed-out CPU thread of my Raspberry Pi 4 NAS transfers. Admittedly, these are mostly "local" drive-to-drive transfers over USB rather than server-to-server transfers over house-wide gigabit. But consider trying the transfer without compression. I'll definitely have to read up on your other flags though, if your rsync works from just one machine rather than both that might solve a minor problem case I have where the destination machine lacks rsync... https://unix.stackexchange.com/questions/188737/does-compression-option-z-with-rsync-speed-up-backup https://unix.stackexchange.com/questions/188737/does-compres...
- nanis 5y ago
  Really good observation. All the settings are from a long time ago when link quality was way worse than what we get these days. I should review those options, definitely. > -C, --cvs-exclude auto-ignore files in the same way CVS does Just goes to show you how long I've been using this stuff :-) The additional options may not be relevant now that I am just deploying creating a new target directory each time.

marstall 5y ago

I'm fully on team spooky admin - however on my latest project, I've been loving being on the free tier of tools like vercel, which deploys from git commit, and firebase, which is mostly maintenance free. That really reduces the amount of devops in my day.

JTbane 5y ago

>you can get away with spooky arcane oldhat sysadmin deploy techniques (scp release to prod hosts, run deploy script, service or cron to keep running) You would be surprised at how many old enterprises still do this. Sure, it's ugly, but it can be simpler than k8s, and anyone with scripting chops can understand it.

wefarrell 5y ago

At this point CI is pretty easy to setup and monitor and it's integrated with practically every development/deployment tool, so I have to disagree about waiting to implement it.

cutemonster 5y ago
What do you use for CI?

NotAnOtter 5y ago

Yeesh I cannot imagine not setting up at least a basic CI. Even if you remove all unit/integration test alarms, a CI is still very useful. The 20 minutes of build -> upload -> deploy script -> check prod vs a 1/2 day to set up a CI and forget it. I'm not sure you save much time but you save a lot of headache

emidln 5y ago
I think you mean automation is very useful. I've worked in a lot of places where we had a single `make deploy` that could compile, test, upload the artifacts, deploy the artifacts to hosts, start the new services, and slowly transition the load balancer to point at the new instances while automatically backing out if error metrics started lighting up. This is just automation, it doesn't have to happen on every commit. Especially when working solo or on a small team, I might not want to maintain a lot of VCS branches and instead, just deploy when I feel it's ready. You have to do all of that stuff anyway, so if you later decide to hook it up to a VCS trigger, you're all set.
- NotAnOtter 5y ago
  I guess that's fair, I'm probably coming into this having only seen it done a certain way and blind to other possibilities. That being said, it would take a lot to convince me a standard CI/CD pipeline is not worth the investment. My only experience with non CI/CD deployment process was a very manual process at a startup, then when I saw my first CI/CD it felt like magic.
  - smackeyacky 5y ago
    A good old makefile is a CI/CD pipeline. All the new fangled tools are re-inventing a lot of capabilities make already has. I blame the new kids on the block who have only ever known Windows :-)