3 ms·

To be more concrete... It isn't acceptable for a service to be running from a years-old build, since it wouldn't pick up any security fixes. Googles infras

To be more concrete... It isn't acceptable for a service to be running from a years-old build, since it wouldn't pick up any security fixes. Googles infrastructure evolves reasonably quickly - so most reasonably large services will need quite a few code changes every year just to keep up with the fact that internal database type X is going away and everyone needs to migrate to database Y which might have subtly different semantics requiring bits of your service be rewritten. There is also a gradual flow of legal/privacy mandates (eg. as of Y date, IP addresses must not flow through technology Z because we now have to consider them personal information) At some point, one of these migrations requires a lot of work, and there aren't enough humans really keen on doing the work, so it gets shut down. Management could assign workers to do the work, but generally maintaining stuff isn't good for the career path, so few volunteer for big but non-impactful projects like this. As soon as the 'founders' of a project have moved on, it is vulnerable to this kind of shutdown.

klodolph 2y ago
We can assume that with minimal staff, the project is still building and deploying new versions with new dependencies. If it weren’t getting new builds and security fixes, it would raise internal alarms designed to catch unpatched software. It’s the “database X is going away” that really, really kills an understaffed project.
brigandish 2y ago
> Googles infrastructure evolves reasonably quickly - so most reasonably large services will need quite a few code changes every year just to keep up with the fact that internal database type X is going away and everyone needs to migrate to database Y which might have subtly different semantics requiring bits of your service be rewritten. I find this an unsatisfying explanation (to be clear, I'm not criticising you but the effect you're describing), if we develop to interfaces as a principle, then underlying implementations shouldn't matter.
- Tyr42 2y ago
  We have interfaces. They have one implementation. Then next year we have different interfaces with a different implementation.
- karatinversion 2y ago
  Only if the interfaces the underlying implementations provide remain stable
- cbhl 2y ago
  Google is where Titus Winters and Hyrum Wright coined "Hyrum's Law". Loosely paraphrased: given enough users, people start depending on the "implicit interface" rather than the defined one. https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
- klodolph 2y ago
  People are responding talking about interfaces changing… I’m going to say that interfaces changing is the whole point. You see weird stuff when you scale to Google scale. I know it’s cliché to talk about “Google scale” but bear with me for a moment. If you are just running some little service, maybe you have a local filesystem and a SQLite database and that’s enough. If you grow, maybe you have a few machines, network storage, and a PostgreSQL server with a hot replica. If you grow, maybe you have distributed storage and you shard your database across multiple servers. If you keep growing, maybe you end up with Google’s situation, where you have filesystems like Colossus and databases like Spanner. These systems didn’t appear overnight. Someone at Google made a distributed database, and after iterations and evolution, we got Spanner, which is now a Google cloud product you can use yourself. There was a lot of interface breakage on the road to getting something like Spanner. Now that Spanner is a Google cloud offering, you can bet your ass that the interface is stable. It just takes a long time to get there, and the internal customers (Google teams building products) don’t want to wait for the stable, final iteration of some infrastructure to arrive. Nobody knows like that looks like until you suffer through the growing pains of building it.
adamc 2y ago
Thanks for the information... That sounds kind of dysfunctional, organizationally. It means well-established services are vulnerable to shutdown because they aren't new and exciting. As a customer, that would make me even less inclined to use Google.

klodolph 2y ago

We can assume that with minimal staff, the project is still building and deploying new versions with new dependencies. If it weren’t getting new builds and security fixes, it would raise internal alarms designed to catch unpatched software. It’s the “database X is going away” that really, really kills an understaffed project.

brigandish 2y ago

> Googles infrastructure evolves reasonably quickly - so most reasonably large services will need quite a few code changes every year just to keep up with the fact that internal database type X is going away and everyone needs to migrate to database Y which might have subtly different semantics requiring bits of your service be rewritten. I find this an unsatisfying explanation (to be clear, I'm not criticising you but the effect you're describing), if we develop to interfaces as a principle, then underlying implementations shouldn't matter.

Tyr42 2y ago
We have interfaces. They have one implementation. Then next year we have different interfaces with a different implementation.
karatinversion 2y ago
Only if the interfaces the underlying implementations provide remain stable
cbhl 2y ago
Google is where Titus Winters and Hyrum Wright coined "Hyrum's Law". Loosely paraphrased: given enough users, people start depending on the "implicit interface" rather than the defined one. https://www.hyrumslaw.com/ https://www.hyrumslaw.com/
klodolph 2y ago
People are responding talking about interfaces changing… I’m going to say that interfaces changing is the whole point. You see weird stuff when you scale to Google scale. I know it’s cliché to talk about “Google scale” but bear with me for a moment. If you are just running some little service, maybe you have a local filesystem and a SQLite database and that’s enough. If you grow, maybe you have a few machines, network storage, and a PostgreSQL server with a hot replica. If you grow, maybe you have distributed storage and you shard your database across multiple servers. If you keep growing, maybe you end up with Google’s situation, where you have filesystems like Colossus and databases like Spanner. These systems didn’t appear overnight. Someone at Google made a distributed database, and after iterations and evolution, we got Spanner, which is now a Google cloud product you can use yourself. There was a lot of interface breakage on the road to getting something like Spanner. Now that Spanner is a Google cloud offering, you can bet your ass that the interface is stable. It just takes a long time to get there, and the internal customers (Google teams building products) don’t want to wait for the stable, final iteration of some infrastructure to arrive. Nobody knows like that looks like until you suffer through the growing pains of building it.

adamc 2y ago

Thanks for the information... That sounds kind of dysfunctional, organizationally. It means well-established services are vulnerable to shutdown because they aren't new and exciting. As a customer, that would make me even less inclined to use Google.