6 ms·

I love the MaxInboundMessageSize example. I've run into that many times. Often there will be a note in the release notes about it, and I know I should read the

I love the MaxInboundMessageSize example. I've run into that many times. Often there will be a note in the release notes about it, and I know I should read the release notes in detail when I upgrade dependencies, but like many people I don't always. Sometimes it's just laziness or complacency -- especially for "utility" libraries like for compression or encoding -- but other times it's a challenge with release notes: * Each version has each release note published independently (or worse: only on the Releases tab in GitHub, and you have to click to expand to read each) * The release notes are really long or dense, and breaking changes are easily missed There's also worse problems: * The release notes don't actually call out the breaking change (you have to read each ticket in detail) * The release notes just say "Bug fixes" or there are no release notes I think along with the suggestions in this article, library authors should also put effort into making good release notes. This includes realizing sometimes people are using from a couple major versions and/or years ago.

majewsky 8y ago
God yes! For the apps that I maintain (and which have users outside my team), I enforce high-quality release notes like you describe. Representative example: https://github.com/sapcc/swift-http-import/blob/master/CHANGELOG.md https://github.com/sapcc/swift-http-import/blob/master/CHANG... (note that this also takes SemVer seriously)
- y0ghur7_xxx 8y ago
  > I enforce high-quality release notes like you describe. Representative example: https://github.com/sapcc/swift-http-import/blob/master/CHANG.. https://github.com/sapcc/swift-http-import/blob/master/CHANG.... (note that this also takes SemVer seriously) Nitpick: you are not using SemVer. A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 -> 1.10.0 -> 1.11.0.[1] [1]https://semver.org/#spec-item-2 https://semver.org/#spec-item-2 Some of your version numbers lack the patch version.
  - majewsky 8y ago
    Thanks for the heads-up. Will fix that in future releases.
CoryG89 8y ago
While it is a good example, you could also use the same example and conclude that the problem was inadequate tests. SemVer is great, but you can't count on dependencies that you do not control actually adhering to it, either intentionally or unintentionally. The only thing that could have prevented something like this for sure was mentioned: > And while nothing in our early testing sent messages larger than 256k, there were plenty of production instances that did. To me, this was the clear failure; not the fact that some dependency broke semver. Their production system relied on being able to send messages larger than 256k, and their tests did not.
- pvorb 8y ago
  But isn't it impractical to test every feature of every library you are using? In an ideal world you would have everything tested in isolation as well as integration. But in practice there will always be a corner case that remains untested because you don't know all internals of the libraries you use.
  - CoryG89 8y ago
    If you want to be able to randomly upgrade those dependencies and not have to worry about a breaking change, then yes. Semver is not going to help you there. Server is only going to help you when someone knows they are releasing breaking change. And even then, only if they are nice enough to actually follow the spec. You don't have to test every bit of every dependency you use, but upgrading them without either carefully reviewing the changes or having tests in place for at least critical functionality is asking for something like this to happen eventually.
- gregmac 8y ago
  While it's easy to say, how far do you go? Do you test every bit of every upstream library you use? The ideal is probably yes, but the reality is this rarely happens. Even with a test, you may not find this. In the IOException example, the author calls out why: > When we upgraded, all our tests passed (because our test fixtures emulated the old behavior and our network was not unstable enough to trigger bad conditions) The only way to catch this type of thing is to emulate the entire network side of things, and that's still only as good as your simulation of the real world. Again, reality is even if you test your upstream to this extent, you're probably mocking a bunch of things, and that may mask something in a way you won't see until possibly production use.

majewsky 8y ago

God yes! For the apps that I maintain (and which have users outside my team), I enforce high-quality release notes like you describe. Representative example: https://github.com/sapcc/swift-http-import/blob/master/CHANGELOG.md https://github.com/sapcc/swift-http-import/blob/master/CHANG... (note that this also takes SemVer seriously)

y0ghur7_xxx 8y ago
> I enforce high-quality release notes like you describe. Representative example: https://github.com/sapcc/swift-http-import/blob/master/CHANG.. https://github.com/sapcc/swift-http-import/blob/master/CHANG.... (note that this also takes SemVer seriously) Nitpick: you are not using SemVer. A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 -> 1.10.0 -> 1.11.0.[1] [1]https://semver.org/#spec-item-2 https://semver.org/#spec-item-2 Some of your version numbers lack the patch version.
- majewsky 8y ago
  Thanks for the heads-up. Will fix that in future releases.

CoryG89 8y ago

While it is a good example, you could also use the same example and conclude that the problem was inadequate tests. SemVer is great, but you can't count on dependencies that you do not control actually adhering to it, either intentionally or unintentionally. The only thing that could have prevented something like this for sure was mentioned: > And while nothing in our early testing sent messages larger than 256k, there were plenty of production instances that did. To me, this was the clear failure; not the fact that some dependency broke semver. Their production system relied on being able to send messages larger than 256k, and their tests did not.

pvorb 8y ago
But isn't it impractical to test every feature of every library you are using? In an ideal world you would have everything tested in isolation as well as integration. But in practice there will always be a corner case that remains untested because you don't know all internals of the libraries you use.
- CoryG89 8y ago
  If you want to be able to randomly upgrade those dependencies and not have to worry about a breaking change, then yes. Semver is not going to help you there. Server is only going to help you when someone knows they are releasing breaking change. And even then, only if they are nice enough to actually follow the spec. You don't have to test every bit of every dependency you use, but upgrading them without either carefully reviewing the changes or having tests in place for at least critical functionality is asking for something like this to happen eventually.
gregmac 8y ago
While it's easy to say, how far do you go? Do you test every bit of every upstream library you use? The ideal is probably yes, but the reality is this rarely happens. Even with a test, you may not find this. In the IOException example, the author calls out why: > When we upgraded, all our tests passed (because our test fixtures emulated the old behavior and our network was not unstable enough to trigger bad conditions) The only way to catch this type of thing is to emulate the entire network side of things, and that's still only as good as your simulation of the real world. Again, reality is even if you test your upstream to this extent, you're probably mocking a bunch of things, and that may mask something in a way you won't see until possibly production use.