5 ms·

I've said it before and I'll say it again: pre-mature infrastructure optimization is the root of all evil. Do me a favor and if you got a startup, stay clear o

I've said it before and I'll say it again: pre-mature infrastructure optimization is the root of all evil. Do me a favor and if you got a startup, stay clear of all this. Everyone wants to reinvent their own flavor of heroku and make your deployment and build pipeline god-awful complex. Their tool of choice? Docker. Before you know it you'll be swimming in containers upon containers. Containers will save us, they'll cry! Meanwhile you have 0 rows of data before you've paid them their first month's salary and have spent time on solving problems of scale you'll never have. Focus on your product, outsource the rest. And leave customized docker setups to mid-stage startups and big corps who already have these problems, or at least the money and people to toil on them. Not everything needs to be a container! And most companies are not and will never be Google!!

deleted 11y ago
[deleted]
kosma 11y ago
Until a few days ago I was a DevOps engineer at a medium-sized tech company. My primary responsibility was dockerizing their applications and infrastructure. I quit the job. The scenario played out just as you said: I ended up single-handedly and poorly re-engineering something that already existed (they did have a working Ansible setup) for no visible gain. "Swimming in containers upon containers" is exactly what happened; they kinda worked, but the farther we got, the more kludges piled on top of each other. In four months work we didn't even hit production - the most we got was a CI/QA service that was actually nothing more than a loose bunch of Python scripts. Between managing dev/test/prod differences, tracing missing logs, removing unused volumes, networking all that stuff together and trying to provide at least a decent level of security, I realized that I'm wasting everyone's time and money. Developers hated it because it filled their workflows with traps and obstacles. Admins hated it because of the lack of tooling. Business hated it because it caused unexplainable delays. The only thing we really accomplished was some compliance with the The Twelve-Factor App - something that could've been done in a week. Hardly a victory. My advice? Forget about Docker unless your primary business is building hosting systems. It will take years before Docker gets mature enough for production, and not without a ton of tooling on top of it and some major architectural changes. Until then, go back to the old UNIX ways of doing things... it worked perfectly since the Epoch and it will continue to work long after the 32-bit time_t rolls over. You'll be fine.
- chasb 11y ago
  We run Docker in production, as a PaaS no less, and this reads like a list of reasons to use a new-generation PaaS.
  - kordless 11y ago
    Thanks for the chocolate again! :)
- Nyetan 11y ago
  Bother you for a little advice? I'm working at a mid-sized tech company and am evaluating Docker for CI, testing and limited, internal deployment usages. The services in question are built with a hodge-podge of shell scripts and build tools, so getting them all to compile locally is a challenge, let alone deploying them. My hope was that containerizing the builds would isolate any configuration problems, and that containerizing the deployed services would cut down on outages by permitting trivial rollbacks (say, by snapshotting all the service containers before each deploy and merely restoring them should a deployment fail). Of course, all of the above could be fixed by traditional means (e.g. rewriting the build system with a single, standard tool; streamlining the deployment process, etc.), but it seemed like Docker could solve 80% of the problems while easing the implementation of the proper solutions down the line. Considering the above, do you still think Docker's a poor fit for business that aren't building hosting systems? Oh, and any nuggets of wisdom you could throw to a newcomer to the industry? :)
  - catern 11y ago
    You don't need Docker to make a standard build environment. You don't need "chroot on steroids", you just need chroot.
    - rmgraham 11y ago
      I've found docker to be "chroot with all your custom fs layouts and custom mount scripts combined" - in other words a much more convenient chroot.
jordanthoms 11y ago
I do agree with you, but we're on Heroku and that also has it's downsides - the biggest one we've found is that CPU performance on 1x and 2x dynos is very unpredictable - We've had to move to Performance dynos ($500/month each!) to get decent performance and prevent random timeouts. Services like Cloud66 are interesting (they manage deployments onto your own EC2 or other cloud infrastructure), but the developer experience doesn't quite match Heroku yet. Heroku really needs some more competition...
- theatraine 11y ago
  Azure App Service (https://azure.microsoft.com/en-us/services/app-service/ https://azure.microsoft.com/en-us/services/app-service/) is Azure's Heroku competitor. It's quite close in features and ease-of-use. Disclosure: I work for this team.
  - jordanthoms 11y ago
    That looks quite nice in the few minutes I spent looking at it. The lack of Ruby support is a issue though, and we're also all on OSX and prefer CLI interfaces for most stuff. Also, one of the killer features of Heroku (which few services seem to replicate) is log drains - I can easily add a http or syslog endpoint and have Heroku send the logs over. The other killer feature which isn't often replicated is One-off dynos, where we can spin up a new instance and get a console attached to it in one command - useful for running database migrations or using Ruby as a CLI to access data. If we were on .NET that would probably be attractive, but it's still not really competing with Heroku.
    - agiamas 11y ago
      you summed up the reasons I use Heroku myself, gz.
stephenr 11y ago
I'm definitely no fan of Docker or the ridiculous containerisation trend, but I think I may disagree with one key thing you said: > Focus on your product, outsource the rest. What do you mean by outsource the rest? Do you mean, "hey we're using AWS <Everything>-as-Service because we don't want to manage a DB cluster or deal with a load balancer? Or do you mean, rely on existing available tools and stop reinventing the wheel every week?
- statico 11y ago
  > What do you mean by outsource the rest? iamleppert means: Identify your company's core competency and do that in house, but outsource or avoid that which is not your core. For example, we're making a game. Gameplay, art, and tech is all done in-house and not with remote contractors because it needs to be -- it's the part of the product we love and the part our players will end up loving. Email, forums, chat, HR, applicant tracking systems, and git hosting are outside of our core and best handled by others.
  - stephenr 11y ago
    Define "handled by others". Installing an exchange server is arguably letting email be "handled by others" because you are not responsible for how it works, just the setup and monitoring, which could be handled by in-house staff or by a contractor. My point is that "focus on the core competency" doesn't have to mean "make our company reliant on a dozen other SaaS businesses who may go offline or change their business model/functionality on a whim"
  - Gigablah 11y ago
    But if you're making a game, and create your own chat service instead, you end up as Slack.
    - statico 11y ago
      We in fact started with an HTML5 group chat application very much like Slack but ended up making a game.
kordless 11y ago
There are reasons other than scale for containerizing your application. Continuous deployments become a bit easier, for example. Reliability, especially during rolling updates, is improved. It's easier to build a zero downtime app that is in continuous deployment with a containerized stack than it is with more traditional instances. Regarding the outsourcing, that's what we're shooting for at Giant Swarm. We've written a stack that runs containers and manages the metal underneath for you. We run the solution as a shared public cluster at giantswarm.io, but can also do private hosted deployments or managed on-prem deployments. It's a complete solution for running containers that feels like a PaaS, but without all the opinionated crap associated with a PaaS. We're basically offering to be your little devops team that could - with containers.

deleted 11y ago

[deleted]

kosma 11y ago

Until a few days ago I was a DevOps engineer at a medium-sized tech company. My primary responsibility was dockerizing their applications and infrastructure. I quit the job. The scenario played out just as you said: I ended up single-handedly and poorly re-engineering something that already existed (they did have a working Ansible setup) for no visible gain. "Swimming in containers upon containers" is exactly what happened; they kinda worked, but the farther we got, the more kludges piled on top of each other. In four months work we didn't even hit production - the most we got was a CI/QA service that was actually nothing more than a loose bunch of Python scripts. Between managing dev/test/prod differences, tracing missing logs, removing unused volumes, networking all that stuff together and trying to provide at least a decent level of security, I realized that I'm wasting everyone's time and money. Developers hated it because it filled their workflows with traps and obstacles. Admins hated it because of the lack of tooling. Business hated it because it caused unexplainable delays. The only thing we really accomplished was some compliance with the The Twelve-Factor App - something that could've been done in a week. Hardly a victory. My advice? Forget about Docker unless your primary business is building hosting systems. It will take years before Docker gets mature enough for production, and not without a ton of tooling on top of it and some major architectural changes. Until then, go back to the old UNIX ways of doing things... it worked perfectly since the Epoch and it will continue to work long after the 32-bit time_t rolls over. You'll be fine.

chasb 11y ago
We run Docker in production, as a PaaS no less, and this reads like a list of reasons to use a new-generation PaaS.
- kordless 11y ago
  Thanks for the chocolate again! :)
Nyetan 11y ago
Bother you for a little advice? I'm working at a mid-sized tech company and am evaluating Docker for CI, testing and limited, internal deployment usages. The services in question are built with a hodge-podge of shell scripts and build tools, so getting them all to compile locally is a challenge, let alone deploying them. My hope was that containerizing the builds would isolate any configuration problems, and that containerizing the deployed services would cut down on outages by permitting trivial rollbacks (say, by snapshotting all the service containers before each deploy and merely restoring them should a deployment fail). Of course, all of the above could be fixed by traditional means (e.g. rewriting the build system with a single, standard tool; streamlining the deployment process, etc.), but it seemed like Docker could solve 80% of the problems while easing the implementation of the proper solutions down the line. Considering the above, do you still think Docker's a poor fit for business that aren't building hosting systems? Oh, and any nuggets of wisdom you could throw to a newcomer to the industry? :)
- catern 11y ago
  You don't need Docker to make a standard build environment. You don't need "chroot on steroids", you just need chroot.
  - rmgraham 11y ago
    I've found docker to be "chroot with all your custom fs layouts and custom mount scripts combined" - in other words a much more convenient chroot.

jordanthoms 11y ago

I do agree with you, but we're on Heroku and that also has it's downsides - the biggest one we've found is that CPU performance on 1x and 2x dynos is very unpredictable - We've had to move to Performance dynos ($500/month each!) to get decent performance and prevent random timeouts. Services like Cloud66 are interesting (they manage deployments onto your own EC2 or other cloud infrastructure), but the developer experience doesn't quite match Heroku yet. Heroku really needs some more competition...

theatraine 11y ago
Azure App Service (https://azure.microsoft.com/en-us/services/app-service/ https://azure.microsoft.com/en-us/services/app-service/) is Azure's Heroku competitor. It's quite close in features and ease-of-use. Disclosure: I work for this team.
- jordanthoms 11y ago
  That looks quite nice in the few minutes I spent looking at it. The lack of Ruby support is a issue though, and we're also all on OSX and prefer CLI interfaces for most stuff. Also, one of the killer features of Heroku (which few services seem to replicate) is log drains - I can easily add a http or syslog endpoint and have Heroku send the logs over. The other killer feature which isn't often replicated is One-off dynos, where we can spin up a new instance and get a console attached to it in one command - useful for running database migrations or using Ruby as a CLI to access data. If we were on .NET that would probably be attractive, but it's still not really competing with Heroku.
  - agiamas 11y ago
    you summed up the reasons I use Heroku myself, gz.

stephenr 11y ago

I'm definitely no fan of Docker or the ridiculous containerisation trend, but I think I may disagree with one key thing you said: > Focus on your product, outsource the rest. What do you mean by outsource the rest? Do you mean, "hey we're using AWS <Everything>-as-Service because we don't want to manage a DB cluster or deal with a load balancer? Or do you mean, rely on existing available tools and stop reinventing the wheel every week?

statico 11y ago
> What do you mean by outsource the rest? iamleppert means: Identify your company's core competency and do that in house, but outsource or avoid that which is not your core. For example, we're making a game. Gameplay, art, and tech is all done in-house and not with remote contractors because it needs to be -- it's the part of the product we love and the part our players will end up loving. Email, forums, chat, HR, applicant tracking systems, and git hosting are outside of our core and best handled by others.
- stephenr 11y ago
  Define "handled by others". Installing an exchange server is arguably letting email be "handled by others" because you are not responsible for how it works, just the setup and monitoring, which could be handled by in-house staff or by a contractor. My point is that "focus on the core competency" doesn't have to mean "make our company reliant on a dozen other SaaS businesses who may go offline or change their business model/functionality on a whim"
- Gigablah 11y ago
  But if you're making a game, and create your own chat service instead, you end up as Slack.
  - statico 11y ago
    We in fact started with an HTML5 group chat application very much like Slack but ended up making a game.

kordless 11y ago

There are reasons other than scale for containerizing your application. Continuous deployments become a bit easier, for example. Reliability, especially during rolling updates, is improved. It's easier to build a zero downtime app that is in continuous deployment with a containerized stack than it is with more traditional instances. Regarding the outsourcing, that's what we're shooting for at Giant Swarm. We've written a stack that runs containers and manages the metal underneath for you. We run the solution as a shared public cluster at giantswarm.io, but can also do private hosted deployments or managed on-prem deployments. It's a complete solution for running containers that feels like a PaaS, but without all the opinionated crap associated with a PaaS. We're basically offering to be your little devops team that could - with containers.