4 ms·

I was the guy at Netflix that had the task of moving the Company from Guava 10 to Guava 11. Guava 11 was backward-incompatible with Guava 10. On the Platform te

I was the guy at Netflix that had the task of moving the Company from Guava 10 to Guava 11. Guava 11 was backward-incompatible with Guava 10. On the Platform team (produced libraries the entire company consumed) we could only move to Guava 11 if EVERYONE moved to Guava 11. The only alternative would have been to shade Guava 11 into the platform library which wasn't pleasant.

BurningFrog 11y ago
That sounds like a real impractical way to run a platform team. Moving the entire company codebase at the same time may be possible with a dozen developers, but when you reach Netflix size it seems impossible. What am I missing?
- Randgalt 11y ago
  It wasn't practical. But that's what we had at the time.
  - hardwaresofton 11y ago
    Just to chime in here -- I think this is a relatively common happening. I've found that companies with lots of teams developing software often look to "standardize" the software being used to create things like web services/pieces of infrastructure. This often starts with creating a local nexus server (in java land), then is followed relatively quickly by creating a "X company commons" for "everyone" to use.
  - BurningFrog 11y ago
    I see :) That kind of problem is tricky, since nobody decided to make it that way. A system worked great when it was instituted, and was never changed, but is now no longer good. Anyway, if/when I become Netflix CTO, I will make sure this is fixed!
foota 11y ago
What is shading a library?
- thecatspaw 11y ago
  I was curious as well, so I went looking: http://stackoverflow.com/a/13620420/4563079 http://stackoverflow.com/a/13620420/4563079 relevant part: "For example, I am developing Foo library, which depends on a specific version (e.g. 1.0) of Bar library. Assuming I cannot make use of other version of Bar lib (because API change, or other technical issues, etc). If I simply declare Bar:1.0 as Foo's dependency in Maven, it is possible to fall into a problem: A Qux project is depending on Foo, and also Bar:2.0 (and it cannot use Bar:1.0 because Qux needs to use new feature in Bar:2.0). Here is the dilemma: should Qux use Bar:1.0 (which Qux's code will not work) or Bar:2.0 (which Foo's code will not work)? In order to solve this problem, developer of Foo can choose to use shade plugin to rename its usage of Bar, so that all classes in Bar:1.0 jar are embedded in Foo jar, and the package of the embedded Bar classes is changed from com.bar to com.foo.bar. By doing so, Qux can safely depends on Bar:2.0 because now Foo is no longer depending on Bar, and it is using is own copy of "altered" Bar located in another package."
  - foota 11y ago
    So it's like name mangling for libraries?
fs111 11y ago
Friends don't let friends use guava.
- Randgalt 11y ago
  Guava is what happens when the owner of a widely used technology is inept. Java is in much better shape today.
Randgalt 11y ago
I should mention that the reason this was such a huge problem was that our dependency system at the time gave each group the option of using the "latest" version of the platform library instead of fixed version. Everyone used it. If, instead, people had locked to a known version of the library it would've been so much easier to upgrade.

BurningFrog 11y ago

That sounds like a real impractical way to run a platform team. Moving the entire company codebase at the same time may be possible with a dozen developers, but when you reach Netflix size it seems impossible. What am I missing?

Randgalt 11y ago
It wasn't practical. But that's what we had at the time.
- hardwaresofton 11y ago
  Just to chime in here -- I think this is a relatively common happening. I've found that companies with lots of teams developing software often look to "standardize" the software being used to create things like web services/pieces of infrastructure. This often starts with creating a local nexus server (in java land), then is followed relatively quickly by creating a "X company commons" for "everyone" to use.
- BurningFrog 11y ago
  I see :) That kind of problem is tricky, since nobody decided to make it that way. A system worked great when it was instituted, and was never changed, but is now no longer good. Anyway, if/when I become Netflix CTO, I will make sure this is fixed!

foota 11y ago

What is shading a library?

thecatspaw 11y ago
I was curious as well, so I went looking: http://stackoverflow.com/a/13620420/4563079 http://stackoverflow.com/a/13620420/4563079 relevant part: "For example, I am developing Foo library, which depends on a specific version (e.g. 1.0) of Bar library. Assuming I cannot make use of other version of Bar lib (because API change, or other technical issues, etc). If I simply declare Bar:1.0 as Foo's dependency in Maven, it is possible to fall into a problem: A Qux project is depending on Foo, and also Bar:2.0 (and it cannot use Bar:1.0 because Qux needs to use new feature in Bar:2.0). Here is the dilemma: should Qux use Bar:1.0 (which Qux's code will not work) or Bar:2.0 (which Foo's code will not work)? In order to solve this problem, developer of Foo can choose to use shade plugin to rename its usage of Bar, so that all classes in Bar:1.0 jar are embedded in Foo jar, and the package of the embedded Bar classes is changed from com.bar to com.foo.bar. By doing so, Qux can safely depends on Bar:2.0 because now Foo is no longer depending on Bar, and it is using is own copy of "altered" Bar located in another package."
- foota 11y ago
  So it's like name mangling for libraries?

fs111 11y ago

Friends don't let friends use guava.

Randgalt 11y ago
Guava is what happens when the owner of a widely used technology is inept. Java is in much better shape today.

Randgalt 11y ago

I should mention that the reason this was such a huge problem was that our dependency system at the time gave each group the option of using the "latest" version of the platform library instead of fixed version. Everyone used it. If, instead, people had locked to a known version of the library it would've been so much easier to upgrade.