4 ms·

Yeah. And it's also not that impressive. We've had what they call "preview environments" for a very long time and at my previous companies too. With GitHub and

Yeah. And it's also not that impressive. We've had what they call "preview environments" for a very long time and at my previous companies too. With GitHub and AWS/Cloud it's trivially easy to do. And like you, we have both "preview environments" and staging and for the exact reason you've highlight. I'm rather skeptical of the company behind this product now since they don't seem to understand the difference and really overselling something that's quite common.

phendrenad2 4y ago
Yes, this is one of those things that was easier before kubernetes. Before: Your developers have a shared Linux box that serves the preview environment. Your ruby app has a tiny loader in front of it that loads the app code from a specific directory based on subdomain. After: You need to route subdomains to different kubernetes containers, and handle deploying new containers when the developer opens a PR (because it's much more work than just copying files onto a server), and you need to handle destroying old containers (because it's much more work than just deleting files on a server). Not to mention, usually kubernetes is managed by "the ops" who don't know anything about the app. Much more difficult to interface with them when it's not "oh one server, and the developers figure it out, and if they royally fuck it up we'll rescue them".
- shamiln 4y ago
  It’s really not that difficult to automate it over helm or something. All you need is a registry, and most cloud services provide a registry, even GitHub has one. Routing domains to a container shouldn’t be significantly different to doing it in production.
  - tracker1 4y ago
    For that matter, you generally will have a single app firewall or ingress server that reverse-proxies to your application anyway.
    - phendrenad2 4y ago
      Helm, registry, firewall, ingress, reverse proxy, these are all concepts that whoever sets up this hypothetical staging environment will have to know about, and will also have to be a developer (in order to care). So basically you need a senior dev to either love kubernetes or learn enough of it to get the job done. This is what I mean by "harder".
echelon 4y ago
I have spent a lot of time thinking about this problem. I would gladly pay for a product that could make a real, honest to god preview environment that you could spin up on-demand and populate it with all of the data you need. This isn't it, and I think it's pretty much an impossible problem to solve without an entirely new engineering culture around testing. How do you duplicate the behavior of the 10 direct data dependencies of your app and the constellation of transitive microservice dependencies? Wingman, Galactus, etc. At a company with 1000+ microservices? And how do you populate it with actual data so you can log in and perform actions? Every single microservice probably needs test data, and it would take a lot of engineering forethought to populate the entire graph. It gets gnarly. I've been on teams at the center of it all - user accounts and login, representation of the core business entities that the rest of the company is built upon. We couldn't even solve the problem for ourselves. There were so many permutations of account states, creating a dedicated API to make test data deviates from a first class API. So how do you coordinate with downstream services to skip IDV (matching SSN), compromised password checks, etc. Perhaps you only bring online a subset of services and fake out the rest. At what level can you fake stuff out? And how do you ensure that the test data works with whatever subset you don't fake? If a single service responds in an unexpected way, it could break or corrupt the state you're relying upon to test. Maybe "preview" works for frontend. Backend, not so much. This is such a massive problem, and I'm only scratching the surface with my description of the kinds of issues you have to solve.
- phendrenad2 4y ago
  Microservices became this mess because people only think about production. Nobody builds microservices as standalone units that should be testable and runnable in isolation, with ZERO external data. Instead, everything relies on everything, so it's just a monolith split up into microrepos. Worse, people have started putting app configuration in S3 and other dumb things (I've seen it...) do you cannot run things locally, or even match production unless you have AWS creds. Madness.
  - TimTheTinker 4y ago
    Microservices should indeed be able to run in isolation, but that doesn't remove the need for end-to-end and integration testing. At a minimum, the product owner should be able to try out the new feature in the UI without having to deal with a staging environment.

phendrenad2 4y ago

Yes, this is one of those things that was easier before kubernetes. Before: Your developers have a shared Linux box that serves the preview environment. Your ruby app has a tiny loader in front of it that loads the app code from a specific directory based on subdomain. After: You need to route subdomains to different kubernetes containers, and handle deploying new containers when the developer opens a PR (because it's much more work than just copying files onto a server), and you need to handle destroying old containers (because it's much more work than just deleting files on a server). Not to mention, usually kubernetes is managed by "the ops" who don't know anything about the app. Much more difficult to interface with them when it's not "oh one server, and the developers figure it out, and if they royally fuck it up we'll rescue them".

shamiln 4y ago
It’s really not that difficult to automate it over helm or something. All you need is a registry, and most cloud services provide a registry, even GitHub has one. Routing domains to a container shouldn’t be significantly different to doing it in production.
- tracker1 4y ago
  For that matter, you generally will have a single app firewall or ingress server that reverse-proxies to your application anyway.
  - phendrenad2 4y ago
    Helm, registry, firewall, ingress, reverse proxy, these are all concepts that whoever sets up this hypothetical staging environment will have to know about, and will also have to be a developer (in order to care). So basically you need a senior dev to either love kubernetes or learn enough of it to get the job done. This is what I mean by "harder".

echelon 4y ago

I have spent a lot of time thinking about this problem. I would gladly pay for a product that could make a real, honest to god preview environment that you could spin up on-demand and populate it with all of the data you need. This isn't it, and I think it's pretty much an impossible problem to solve without an entirely new engineering culture around testing. How do you duplicate the behavior of the 10 direct data dependencies of your app and the constellation of transitive microservice dependencies? Wingman, Galactus, etc. At a company with 1000+ microservices? And how do you populate it with actual data so you can log in and perform actions? Every single microservice probably needs test data, and it would take a lot of engineering forethought to populate the entire graph. It gets gnarly. I've been on teams at the center of it all - user accounts and login, representation of the core business entities that the rest of the company is built upon. We couldn't even solve the problem for ourselves. There were so many permutations of account states, creating a dedicated API to make test data deviates from a first class API. So how do you coordinate with downstream services to skip IDV (matching SSN), compromised password checks, etc. Perhaps you only bring online a subset of services and fake out the rest. At what level can you fake stuff out? And how do you ensure that the test data works with whatever subset you don't fake? If a single service responds in an unexpected way, it could break or corrupt the state you're relying upon to test. Maybe "preview" works for frontend. Backend, not so much. This is such a massive problem, and I'm only scratching the surface with my description of the kinds of issues you have to solve.

phendrenad2 4y ago
Microservices became this mess because people only think about production. Nobody builds microservices as standalone units that should be testable and runnable in isolation, with ZERO external data. Instead, everything relies on everything, so it's just a monolith split up into microrepos. Worse, people have started putting app configuration in S3 and other dumb things (I've seen it...) do you cannot run things locally, or even match production unless you have AWS creds. Madness.
- TimTheTinker 4y ago
  Microservices should indeed be able to run in isolation, but that doesn't remove the need for end-to-end and integration testing. At a minimum, the product owner should be able to try out the new feature in the UI without having to deal with a staging environment.