3 ms·

If you spend $100K on a staging environment and it doesn't give you confidence that it won't fail on production, the problem is Your Staging, not The Staging.

If you spend $100K on a staging environment and it doesn't give you confidence that it won't fail on production, the problem is Your Staging, not The Staging. If we have something fail in production, the first question we ask ourselves is: Why didn't we catch this in staging? 90% of the time, the answer is: "We didn't test if properly but thought we did." Our rate of patching for defect is rare. As far as the cost, our staging is run in hardware retired from production, with some upgrades (mostly RAM and SSDs). I'm not sure we've spent the authors $100K over 2 decades, let alone in a single year. But YMMV.

hinkley 6y ago
Everybody is in the cloud now, where there are no hand-me-downs, or at least none we get to use.
- nullify88 6y ago
  When AWS releases a new instance generation, everyone flocks to them. For previous generations theres usually ample capacity to run as spot instances uninterrupted over long periods of time, giving your staging environment significant savings. I'd call that a hand me down.
  - hinkley 6y ago
    Except the top level comment was doubting they ever spent 100k total in the history of their staging system. That’s not going to be true for AWS even with the nosebleed seats, is it? And are you really going to run your entire staging environment in spot instances? What happens on peak days?
    - linsomniac 6y ago
      I didn't mean to imply that the author couldn't have spent $100K. There are clearly situations where an expensive staging environment would be needed. I was just saying that we've been able to do our staging for a fraction of that price, and had good results.
- birdyrooster 6y ago
  A year and a half ago, IBM did a study and found that most companies are only 20 percent to cloud transformation. The reality is that there is a ridiculous amount of private datacenters and iron around. https://www.ibm.com/blogs/cloud-computing/2019/03/05/20-percent-cloud-transformation/ https://www.ibm.com/blogs/cloud-computing/2019/03/05/20-perc...
  - hbogert 6y ago
    This makes it sound like 100% is the goal. Some companies don't get added value by going to the cloud. The cloud isn't cheap for example (among many others) for storage heavy workloads. And sometimes it's also a regulatory thing.
    - birdyrooster 6y ago
      Thank you for saying this, I am more than frustrated by that mentality whenever I go to conferences or message boards. There’s a big world out there and most of does not resemble the practices of internet companies. It’s this fallacy of inevitability for cloud.
- skissane 6y ago
  Once I visited a customer site who were having a performance problem. They ran everything on AWS. The day before I arrived on-site, they spun up a brand new production-sized test environment just for me to debug their issue in. I spent a day or two on-site working out what was causing their performance problem. When I left, they deleted that environment. They only paid for its existence for 2-3 days. So with cloud you can get away from fixed always-on staging environments, and move to "staging environments on demand". (My recollection was they loved everything about AWS except the price tag, although creating a personal staging environment for me to debug their issue in would have only made a marginal difference to that total cost.)

hinkley 6y ago

Everybody is in the cloud now, where there are no hand-me-downs, or at least none we get to use.

nullify88 6y ago
When AWS releases a new instance generation, everyone flocks to them. For previous generations theres usually ample capacity to run as spot instances uninterrupted over long periods of time, giving your staging environment significant savings. I'd call that a hand me down.
- hinkley 6y ago
  Except the top level comment was doubting they ever spent 100k total in the history of their staging system. That’s not going to be true for AWS even with the nosebleed seats, is it? And are you really going to run your entire staging environment in spot instances? What happens on peak days?
  - linsomniac 6y ago
    I didn't mean to imply that the author couldn't have spent $100K. There are clearly situations where an expensive staging environment would be needed. I was just saying that we've been able to do our staging for a fraction of that price, and had good results.
birdyrooster 6y ago
A year and a half ago, IBM did a study and found that most companies are only 20 percent to cloud transformation. The reality is that there is a ridiculous amount of private datacenters and iron around. https://www.ibm.com/blogs/cloud-computing/2019/03/05/20-percent-cloud-transformation/ https://www.ibm.com/blogs/cloud-computing/2019/03/05/20-perc...
- hbogert 6y ago
  This makes it sound like 100% is the goal. Some companies don't get added value by going to the cloud. The cloud isn't cheap for example (among many others) for storage heavy workloads. And sometimes it's also a regulatory thing.
  - birdyrooster 6y ago
    Thank you for saying this, I am more than frustrated by that mentality whenever I go to conferences or message boards. There’s a big world out there and most of does not resemble the practices of internet companies. It’s this fallacy of inevitability for cloud.
skissane 6y ago
Once I visited a customer site who were having a performance problem. They ran everything on AWS. The day before I arrived on-site, they spun up a brand new production-sized test environment just for me to debug their issue in. I spent a day or two on-site working out what was causing their performance problem. When I left, they deleted that environment. They only paid for its existence for 2-3 days. So with cloud you can get away from fixed always-on staging environments, and move to "staging environments on demand". (My recollection was they loved everything about AWS except the price tag, although creating a personal staging environment for me to debug their issue in would have only made a marginal difference to that total cost.)