5 ms·

The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted. The Linux

by devit 11y ago

The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted. The Linux kernel generally uses this policy for internal APIs for example.

Lewisham 11y ago
We do, mostly. Because Piper is a global repository, we have systems to do global safe refactors, and do so often. If the API changes drastically, there's usually a lengthy deprecation period before the API is switched over.
revelation 11y ago
I guess that works for the Linux kernel, but I would presume that for a large distributed operation like Google it would be much better to simply deprecate/version APIs and have the project teams update to a deadline. I mean, it's presumably impossible to have a single computer running a single OS build all of the Google software and run the testing.
- teraflop 11y ago
  Google is rumored to have an extremely powerful distributed compilation farm. It wouldn't surprise me if a single developer could make a change that affects the entire codebase and test it themselves.
  - malkia 11y ago
    The key to this in her talk was TAP
stock_toaster 11y ago
> The solution to the excessive API change problem is to force > whoever changes the API to fix all the consumers himself > before the change is accepted. This doesn't seem scalable. Let's consider the case of one api endpoint being changed by one developer, to add a new param to a function call. Further assume that this impacts hundreds of projects. Does it really make sense to make one developer update those hundred projects? Not only will it take forever for it to get finished (possibly never if there are new consumers of this api coming online frequently), but the developer of the core api may not have any experience in the impacted consumers of this codebase. I think the end result of this policy would be nothing once written ever would get updated, and new apis would just be added all the time (api explosion).
- masterj 11y ago
  It's scalable with the right tools. If you can write a transform at the level of the AST that will make the change for you, you can do it in one commit. FB has written about this: https://medium.com/@cpojer/effective-javascript-codemods-5a6686bb46fb https://medium.com/@cpojer/effective-javascript-codemods-5a6... Not that it's a silver bullet, but it can make a lot of these cases non-issues.
- XorNot 11y ago
  Google's product explosions / surprise deprecations possibly hint that this is what happens? Changing the API becomes cumbersome, so you just make a new product with a new API to do an end-run around the requirement...
- zenbowman 11y ago
  If all your RPCs take a single parameter, this isn't a problem provided you use universal defaults.
- nulltype 11y ago
  It seems bad but the alternative (wait forever for groups that don't understand the change and may not even exist anymore to update their code) seems worse.
- thrownaway2424 11y ago
  It maybe isn't scalable, but that's part of the benefit. If you want to make a change to a widely-used API, it's going to be a lot of work, and it's not going to be a lot of work for the users of the API, it's going to be a lot of work for _you_ because _you_ are required to do it yourself. This prevents a lot of API churn unless the benefit is clear and sufficiently large. If it was any other way you'd rapidly reach a useless equilibrium where random engineers were demanding that thousands of other engineers fulfill unfunded mandates for what might turn out to be negligible benefits.
  - philwelch 11y ago
    That's one extreme. Another extreme is that you have the API versioning from hell, where you can never get rid of technical debt because any and all API changes will break someone, somewhere, who has no reason to migrate, so you're left keeping ancient code on life support indefinitely.
trollian 11y ago
This makes it harder to change APIs. Which has advantages and disadvantages.
Touche 11y ago
> The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted. Having people unaware of a project's purpose making changes to its code sounds like a nightmare to me.
- sowbug 11y ago
  When that's the cultural norm, people adjust accordingly. Tools include liberal use of assertions, defensive tests, and most important, code reviews that catch and remove quirkiness. It's a nice environment to work in. In addition to hastening Noogler onboarding, it also increases employee retention. If you are an expert in your project's codebase but get burned out, you can easily transfer to another project and be almost immediately productive. Obviously, there's domain-specific knowledge that doesn't transfer easily or quickly from project to project. But that's quite different from self-inflicted code fragility; one's an asset and the other's a liability.

by devit 11y ago

Lewisham 11y ago

We do, mostly. Because Piper is a global repository, we have systems to do global safe refactors, and do so often. If the API changes drastically, there's usually a lengthy deprecation period before the API is switched over.

revelation 11y ago

I guess that works for the Linux kernel, but I would presume that for a large distributed operation like Google it would be much better to simply deprecate/version APIs and have the project teams update to a deadline. I mean, it's presumably impossible to have a single computer running a single OS build all of the Google software and run the testing.

teraflop 11y ago
Google is rumored to have an extremely powerful distributed compilation farm. It wouldn't surprise me if a single developer could make a change that affects the entire codebase and test it themselves.
- malkia 11y ago
  The key to this in her talk was TAP

stock_toaster 11y ago

> The solution to the excessive API change problem is to force > whoever changes the API to fix all the consumers himself > before the change is accepted. This doesn't seem scalable. Let's consider the case of one api endpoint being changed by one developer, to add a new param to a function call. Further assume that this impacts hundreds of projects. Does it really make sense to make one developer update those hundred projects? Not only will it take forever for it to get finished (possibly never if there are new consumers of this api coming online frequently), but the developer of the core api may not have any experience in the impacted consumers of this codebase. I think the end result of this policy would be nothing once written ever would get updated, and new apis would just be added all the time (api explosion).

masterj 11y ago
It's scalable with the right tools. If you can write a transform at the level of the AST that will make the change for you, you can do it in one commit. FB has written about this: https://medium.com/@cpojer/effective-javascript-codemods-5a6686bb46fb https://medium.com/@cpojer/effective-javascript-codemods-5a6... Not that it's a silver bullet, but it can make a lot of these cases non-issues.
XorNot 11y ago
Google's product explosions / surprise deprecations possibly hint that this is what happens? Changing the API becomes cumbersome, so you just make a new product with a new API to do an end-run around the requirement...
zenbowman 11y ago
If all your RPCs take a single parameter, this isn't a problem provided you use universal defaults.
nulltype 11y ago
It seems bad but the alternative (wait forever for groups that don't understand the change and may not even exist anymore to update their code) seems worse.
thrownaway2424 11y ago
It maybe isn't scalable, but that's part of the benefit. If you want to make a change to a widely-used API, it's going to be a lot of work, and it's not going to be a lot of work for the users of the API, it's going to be a lot of work for _you_ because _you_ are required to do it yourself. This prevents a lot of API churn unless the benefit is clear and sufficiently large. If it was any other way you'd rapidly reach a useless equilibrium where random engineers were demanding that thousands of other engineers fulfill unfunded mandates for what might turn out to be negligible benefits.
- philwelch 11y ago
  That's one extreme. Another extreme is that you have the API versioning from hell, where you can never get rid of technical debt because any and all API changes will break someone, somewhere, who has no reason to migrate, so you're left keeping ancient code on life support indefinitely.

trollian 11y ago

This makes it harder to change APIs. Which has advantages and disadvantages.

Touche 11y ago

> The solution to the excessive API change problem is to force whoever changes the API to fix all the consumers himself before the change is accepted. Having people unaware of a project's purpose making changes to its code sounds like a nightmare to me.

sowbug 11y ago
When that's the cultural norm, people adjust accordingly. Tools include liberal use of assertions, defensive tests, and most important, code reviews that catch and remove quirkiness. It's a nice environment to work in. In addition to hastening Noogler onboarding, it also increases employee retention. If you are an expert in your project's codebase but get burned out, you can easily transfer to another project and be almost immediately productive. Obviously, there's domain-specific knowledge that doesn't transfer easily or quickly from project to project. But that's quite different from self-inflicted code fragility; one's an asset and the other's a liability.