4 ms·

Not a Rust user but this all sounds remarkably painful. Is it common? The only other compiled, type-safe language I use regularly is C# (via Unity) and I don't

Not a Rust user but this all sounds remarkably painful. Is it common? The only other compiled, type-safe language I use regularly is C# (via Unity) and I don't recall this level of upheaval.

ViViDboarder 6y ago
I’m new to Rust, but it doesn’t appear to be. If I’m following correctly, it would only occur if a library is using a struct defined in a dependency in a public API you are using and you also use the same dependency elsewhere in your application. Upgrading the version elsewhere in your application could break passing that struct. Most of the time I see public APIs which either base types or struct a defined by the library itself.
deleted 6y ago
[deleted]
mbrubeck 6y ago
This is uncommon, because it's not really necessary except for “foundational” libraries that are used “publicly” (i.e., not just internally) by hundreds of other libraries. If only three or four of my dependencies use the Foo crate, then I can just wait for all of them to upgrade to Foo 2.0, or I can do it myself and submit pull requests. In this case, I don't really care whether Foo's maintainer uses the semver trick. However, if I'm developing a really big project like Servo or rustc, and Foo is a foundational crate that is used by dozens of my (transitive) dependencies, then waiting for all of them to upgrade (or doing the work to upgrade all of them myself) starts to become prohibitive. Now a bit of work by Foo's maintainer can save a lot of time for large downstream consumers. An example of a “foundational” crate that has used the semver trick is `num-traits`, which has over a thousand reverse dependencies: https://crates.io/crates/num-traits/0.1.43 https://crates.io/crates/num-traits/0.1.43
geofft 6y ago
It's only really relevant when a) you release semver-incompatible upgrades (e.g., 1.x to 2.x), b) people use types from your library in their APIs, i.e., their types depend on yours, and c) your library is sufficiently widely used that people are likely to have transitive dependencies on it by multiple routes, such that a couple of their direct dependencies will have upgraded to your new version and a couple wouldn't have gotten around to it. The 'libc' crate was a perfect storm of all three. It was still at 0.x, and the whole point of the library is to define types/bindings for others to use. It also requires the FOSS model of independent owners, I guess, since that's why some consumers haven't upgraded. Inside a company and especially in a monorepo, someone - probably the person making the breaking change - would have just upgraded everyone at once. And a redistributor of FOSS like a Linux distro would have probably just patched everyone at once or held back upgrades until they could - in fact, this is the same shape of problem as upgrading from OpenSSL 1.0 to 1.1, or GTK 2 to 3, or libstdc++'s new ABI, or whatever.
- yjftsjthsd-h 6y ago
  > Inside a company and especially in a monorepo, someone - probably the person making the breaking change - would have just upgraded everyone at once. Ideally yes, in practice not always. The easy way to break it is of course political problems, but it's perfectly possible to be stuck in a situation where unfortunate design decisions early on leave you stuck unable to make breaking changes even if everyone is on the same side (at any rate, not without taking a massive service disruption that would violate our contracts with customers).
  - samus 6y ago
    At work we had that situation with Spring. In the far future, we would like to upgrade to Java > 8, which makes an upgrade to Spring 5 necessary. But the frontend module uses Spring Flow Portlets, which are deprecated and were removed in Spring 5. This was not really a concern though, until too many security issues were found that made the upgrade necessary. In the end we settled on leaving the frontend on Spring 4 and upgrading everything else. It is very ugly, but the only real issue so far is that Spring 4 cannot parse Spring MVC 5's REST encoding of exceptions. Fortunately, we will build a new frontend this year...
1wd 6y ago
C# has assembly binding redirection. https://stackoverflow.com/a/43366172/3679043 https://stackoverflow.com/a/43366172/3679043
- emn13 6y ago
  Yeah, and they're a huge pain if you indeed actually need to use them, and often result in a non-working mess. This wasn't all that uncommon in the early days of .net core, which was particularly bad at this IIRC largely because many foundational libraries were split into packages that essentially could only ever be upgraded in concert. There are a few technical nuances that mean I'm sure this isn't quite the same as the rust case, but it's pretty bad nontheless. Even well thought-out transitions like .net standard weren't free from gotcha's particularly when mixed with multitargeting and deep transitive dependency graphs (which was pretty easy to get in the early .net core days). The whole thing is clearly not great (not just in C#). In particular - many of these problems are entirely artificial. If the type system were sufficiently dynamic, or magically psychic - the problems often go away. E.g. in the rust example - void_c didn't actually change, it was just incompatible because rustc isn't psychic. Even seemingly serious problems like an interface (~ in rust a trait) gaining a new method aren't necessarily fundamentally breaking - as long as you're not calling it. Even truly breaking changes like "best-solution-finder returns a different result and has an incompatible API" might not be breaking between in the case of a diamond dependency pattern as long as the method's side effects allow running both versions - one for each transitive dependency requesting that specific version. Even cases where the break is "real" like a rename, it'd be trivial to consider a shim that allows old consumers to use the new api. In the vast, vast majority of cases in my experience this kind of breakage is a problem due to technicalities. That doesn't mean I have a clue how to solve that, but it does beg the question: isn't it possible for a dependency resolution system to do fundamentally better, here?
  - whoisthemachine 6y ago
    In C#, default implementations of new methods on interfaces should at least reduce the pain of additive changes.
  - 6y ago

Not a Rust user but this all sounds remarkably painful. Is it common? The only other compiled, type-safe language I use regularly is C# (via Unity) and I don't recall this level of upheaval.

ViViDboarder 6y ago

I’m new to Rust, but it doesn’t appear to be. If I’m following correctly, it would only occur if a library is using a struct defined in a dependency in a public API you are using and you also use the same dependency elsewhere in your application. Upgrading the version elsewhere in your application could break passing that struct. Most of the time I see public APIs which either base types or struct a defined by the library itself.

deleted 6y ago

[deleted]

mbrubeck 6y ago

This is uncommon, because it's not really necessary except for “foundational” libraries that are used “publicly” (i.e., not just internally) by hundreds of other libraries. If only three or four of my dependencies use the Foo crate, then I can just wait for all of them to upgrade to Foo 2.0, or I can do it myself and submit pull requests. In this case, I don't really care whether Foo's maintainer uses the semver trick. However, if I'm developing a really big project like Servo or rustc, and Foo is a foundational crate that is used by dozens of my (transitive) dependencies, then waiting for all of them to upgrade (or doing the work to upgrade all of them myself) starts to become prohibitive. Now a bit of work by Foo's maintainer can save a lot of time for large downstream consumers. An example of a “foundational” crate that has used the semver trick is `num-traits`, which has over a thousand reverse dependencies: https://crates.io/crates/num-traits/0.1.43 https://crates.io/crates/num-traits/0.1.43

geofft 6y ago

It's only really relevant when a) you release semver-incompatible upgrades (e.g., 1.x to 2.x), b) people use types from your library in their APIs, i.e., their types depend on yours, and c) your library is sufficiently widely used that people are likely to have transitive dependencies on it by multiple routes, such that a couple of their direct dependencies will have upgraded to your new version and a couple wouldn't have gotten around to it. The 'libc' crate was a perfect storm of all three. It was still at 0.x, and the whole point of the library is to define types/bindings for others to use. It also requires the FOSS model of independent owners, I guess, since that's why some consumers haven't upgraded. Inside a company and especially in a monorepo, someone - probably the person making the breaking change - would have just upgraded everyone at once. And a redistributor of FOSS like a Linux distro would have probably just patched everyone at once or held back upgrades until they could - in fact, this is the same shape of problem as upgrading from OpenSSL 1.0 to 1.1, or GTK 2 to 3, or libstdc++'s new ABI, or whatever.

yjftsjthsd-h 6y ago
> Inside a company and especially in a monorepo, someone - probably the person making the breaking change - would have just upgraded everyone at once. Ideally yes, in practice not always. The easy way to break it is of course political problems, but it's perfectly possible to be stuck in a situation where unfortunate design decisions early on leave you stuck unable to make breaking changes even if everyone is on the same side (at any rate, not without taking a massive service disruption that would violate our contracts with customers).
- samus 6y ago
  At work we had that situation with Spring. In the far future, we would like to upgrade to Java > 8, which makes an upgrade to Spring 5 necessary. But the frontend module uses Spring Flow Portlets, which are deprecated and were removed in Spring 5. This was not really a concern though, until too many security issues were found that made the upgrade necessary. In the end we settled on leaving the frontend on Spring 4 and upgrading everything else. It is very ugly, but the only real issue so far is that Spring 4 cannot parse Spring MVC 5's REST encoding of exceptions. Fortunately, we will build a new frontend this year...

1wd 6y ago

C# has assembly binding redirection. https://stackoverflow.com/a/43366172/3679043 https://stackoverflow.com/a/43366172/3679043

emn13 6y ago
Yeah, and they're a huge pain if you indeed actually need to use them, and often result in a non-working mess. This wasn't all that uncommon in the early days of .net core, which was particularly bad at this IIRC largely because many foundational libraries were split into packages that essentially could only ever be upgraded in concert. There are a few technical nuances that mean I'm sure this isn't quite the same as the rust case, but it's pretty bad nontheless. Even well thought-out transitions like .net standard weren't free from gotcha's particularly when mixed with multitargeting and deep transitive dependency graphs (which was pretty easy to get in the early .net core days). The whole thing is clearly not great (not just in C#). In particular - many of these problems are entirely artificial. If the type system were sufficiently dynamic, or magically psychic - the problems often go away. E.g. in the rust example - void_c didn't actually change, it was just incompatible because rustc isn't psychic. Even seemingly serious problems like an interface (~ in rust a trait) gaining a new method aren't necessarily fundamentally breaking - as long as you're not calling it. Even truly breaking changes like "best-solution-finder returns a different result and has an incompatible API" might not be breaking between in the case of a diamond dependency pattern as long as the method's side effects allow running both versions - one for each transitive dependency requesting that specific version. Even cases where the break is "real" like a rename, it'd be trivial to consider a shim that allows old consumers to use the new api. In the vast, vast majority of cases in my experience this kind of breakage is a problem due to technicalities. That doesn't mean I have a clue how to solve that, but it does beg the question: isn't it possible for a dependency resolution system to do fundamentally better, here?
- whoisthemachine 6y ago
  In C#, default implementations of new methods on interfaces should at least reduce the pain of additive changes.
- 6y ago