3 ms·

> I am again aghast at how complex it is -- not to mention that there isn't much 'force' to push you to learn how to recover from mistakes in production (leadin

> I am again aghast at how complex it is -- not to mention that there isn't much 'force' to push you to learn how to recover from mistakes in production (leading to more downtime than I ever saw with one environment). It's interesting that most of the responses say we moved to multiple environments "because production matters". Yet, your experience says that a single environment is leads to less downtime in production, not more. Is there any actual data on this in general? E.g., some study of downtime in single vs multiple environments? So much of software engineering "best practices" seem to amount to littler more than herd opinion, but rarely have anything substantive to back them up except somebodies experience.

renewiltord 4y ago
I don't believe there is concrete evidence on the multiple environment thing but I think it's likely to come from what the environments are used for. For instance, Netflix's approach plus the Dev Ops Handbook and the Phoenix Project all push frequent deploys as a win. Personally I have found that to be the case as well. However developing in production is like git pushing every keystroke. Honestly, doesn't yield any benefit. So personally I like local iteration, and then trying it out as a canary, and then go to prod. The canary is in the prod environment. In the end, though, I say all this and the levels.io guy writes his stuff right on the VPS sometimes and absolutely slays so who am I to prescribe.
pcthrowaway 4y ago
I think the size of the team would be a big factor too. One to three people (maybe) working on a web app can develop against prod and ship right to prod without breaking things too badly, 99% of the time. Once you have >10 people working on a product, and committing changes in parallel, I'd wager it's impossible to avoid breakages, even with dev/prod/test environments
- withinboredom 4y ago
  We had ~50-100 devs working on that system on any given day ... but only around 5-10 merges per day. However, each team was responsible for a given area of the code (through convention, not any hard rules), so in reality, each area of code was only touched by 2-3 people at a time.

renewiltord 4y ago

I don't believe there is concrete evidence on the multiple environment thing but I think it's likely to come from what the environments are used for. For instance, Netflix's approach plus the Dev Ops Handbook and the Phoenix Project all push frequent deploys as a win. Personally I have found that to be the case as well. However developing in production is like git pushing every keystroke. Honestly, doesn't yield any benefit. So personally I like local iteration, and then trying it out as a canary, and then go to prod. The canary is in the prod environment. In the end, though, I say all this and the levels.io guy writes his stuff right on the VPS sometimes and absolutely slays so who am I to prescribe.

pcthrowaway 4y ago

I think the size of the team would be a big factor too. One to three people (maybe) working on a web app can develop against prod and ship right to prod without breaking things too badly, 99% of the time. Once you have >10 people working on a product, and committing changes in parallel, I'd wager it's impossible to avoid breakages, even with dev/prod/test environments

withinboredom 4y ago
We had ~50-100 devs working on that system on any given day ... but only around 5-10 merges per day. However, each team was responsible for a given area of the code (through convention, not any hard rules), so in reality, each area of code was only touched by 2-3 people at a time.