5 ms·

That's why developing with shared libraries is such a PITA. Got to preserve those precious ABIs.

viraptor 10y ago
What exactly do you mean by PITA? In most cases in my experience, you have two very simple choices - 1. don't modify the existing API / public structures as you change things and the ABI will stay safe; 2. just change things and bump up major number when necessary. The third way of carefully doing only additive modifications to the library to preserve ABI even over upgrades is hard - that's true. But I don't think it's needed that often once most of the features are in. And again - it applies only if you've got some really popular software that gains anything by doing it this way, rather than just bumping up the version.
- majewsky 10y ago
  This. Working with SO versions is much easier in my experience than working with Ruby gems where every minor upgrade could (and sometimes would) break everything. So with shared library dependencies, you can be very broad (e.g. "any libm.so.5 please") and get security updates etc. automatically without needing to recompile, while in "modern" programming languages you will be locked onto old and rancid libraries because the app developer doesn't bother updating the locks.
  - dunkelheit 10y ago
    I think you are talking about compatibility in general (which is great to have!) but not about ABI compatibility. Nothing apart from cultural conventions prevents maintainer of a ruby gem from strictly adhering to semver or maintainer of a C library from breaking everything in a minor version release.
    - majewsky 10y ago
      Agreed. Ruby just hasn't developed the culture of compatibility that's mostly adhered to e.g. in Qt/KDE land where I lived for a long time. Related: I'd like it very much if compatibility was enforced automatically. For example, if I publish version 1.2 of my library to $repository, then $repository checks if 1.2 passes the 1.2 tests (of course), and also if it passes the 1.1 and 1.0 tests. If any of these tests fail, then release should be downright refused unless you upload it as 2.0. And of course, once you have that, you can think of any check you like. In C++, you can pretty reliably detect code changes that change your ABI in backwards-compatible or -incompatible ways, so you could allow backwards-incompatible changes only in major and backwards-compatible changes only in minor releases.
  - digi_owl 10y ago
    And then distro package managers go about and complicate things by having overly rigid dependencies and only allowing one version of each package name to be installed. And the recently introduced container based replacements are and exercise in shooting twee twee birds with AAA...
    - lwf 10y ago
      Sure, which is why we have `libopenssl0.9.8`, `libopenssl1.0.0`, etc. In Debian at least, the policy is that package names for libraries should include something approximating the SONAME to allow for exactly this. Eventually you drop the old one, because you can no longer provide security support or for some other reason. But if there are compelling reasons otherwise, distributions can and do provide multiple ABIs for the same library.
      - digi_owl 10y ago
        And that just exasperates the problem.
        Pxtl 10y ago
        At the very least shoulnd't they be providing and supporting multiple major versions on the same system? Major versions are, by definition, for breaking/incompatible changes, so having them being mutually exclusive creates impossible situations doesn't it?
        digi_owl 10y ago
        Well yes, but using the package name to distinguish them makes it harder for third parties to do packages. This is the complaint that has lead to the development of container based formats in the first place. Thing is that with sonames the actual files are distinguished anyways, so mangling the package names to get around collisions are just an artifact of an overly rigid package manager.
- dunkelheit 10y ago
  What I mean is that in addition to rules of the language I program in (say C or C++) there is another bunch of seemingly arbitrary rules that I must constantly be aware of to ensure nothing breaks. Rules like "don't reorder fields of structs" or "you can add new methods to a class as long as they are not virtual. and sometimes you can add virtual methods too as long as they end up at the end of vtable". How do I ensure that I conform to these rules apart from being disciplined about them? If everything compiles, am I good? No! If everything links, am I good? No! Depending on linker options and the nature of incompatibility the program using incompatible ABIs can blow up at runtime or just silently corrupt data. And don't get me started on the venerable "pimpl idiom". A page of boilerplate just to ensure the most basic thing. Hope that clarifies my short sentiment a bit :) I agree that once you grasp the rules following them is not that hard but it is just another bit of incidental complexity that we agreed to maintain.
  - viraptor 10y ago
    I think most of the issues you listed are quite easy to check automatically. For example http://ispras.linuxbase.org/index.php/ABI_compliance_checker http://ispras.linuxbase.org/index.php/ABI_compliance_checker or anything in the "similar software" section. Of course it won't save you from business logic bugs (changing the meaning, but not size/location of a value), but "how do I ensure that I conform" is 90+% possible to verify automatically.
    - dunkelheit 10y ago
      Thanks, didn't know about it! Still, that's a 3-rd party tool and I still have to write my pimpls. I stand by my PITA sentiment :)
  - rectang 10y ago
    C++ ABI compat rules really hurt when they prevent you from evolving a library in a sensible way. Say that you really need to add a method to a virtual class. Unless you planned ahead by adding placeholder methods, you're SOL. What a lousy design constraint! This problem is solvable at the language/runtime level by resolving vtable offsets at runtime, but that doesn't help the existing C++ userbase.
ascotan 10y ago
C/C++ shared libraries are very easy to work with and distribute. Unfortunately this is becoming a lost skill in the world of javascript. People inherently thing 'old = bad' and are doom to reinvent the same thing over and over.

What exactly do you mean by PITA? In most cases in my experience, you have two very simple choices - 1. don't modify the existing API / public structures as you change things and the ABI will stay safe; 2. just change things and bump up major number when necessary. The third way of carefully doing only additive modifications to the library to preserve ABI even over upgrades is hard - that's true. But I don't think it's needed that often once most of the features are in. And again - it applies only if you've got some really popular software that gains anything by doing it this way, rather than just bumping up the version.

majewsky 10y ago
This. Working with SO versions is much easier in my experience than working with Ruby gems where every minor upgrade could (and sometimes would) break everything. So with shared library dependencies, you can be very broad (e.g. "any libm.so.5 please") and get security updates etc. automatically without needing to recompile, while in "modern" programming languages you will be locked onto old and rancid libraries because the app developer doesn't bother updating the locks.
- dunkelheit 10y ago
  I think you are talking about compatibility in general (which is great to have!) but not about ABI compatibility. Nothing apart from cultural conventions prevents maintainer of a ruby gem from strictly adhering to semver or maintainer of a C library from breaking everything in a minor version release.
  - majewsky 10y ago
    Agreed. Ruby just hasn't developed the culture of compatibility that's mostly adhered to e.g. in Qt/KDE land where I lived for a long time. Related: I'd like it very much if compatibility was enforced automatically. For example, if I publish version 1.2 of my library to $repository, then $repository checks if 1.2 passes the 1.2 tests (of course), and also if it passes the 1.1 and 1.0 tests. If any of these tests fail, then release should be downright refused unless you upload it as 2.0. And of course, once you have that, you can think of any check you like. In C++, you can pretty reliably detect code changes that change your ABI in backwards-compatible or -incompatible ways, so you could allow backwards-incompatible changes only in major and backwards-compatible changes only in minor releases.
- digi_owl 10y ago
  And then distro package managers go about and complicate things by having overly rigid dependencies and only allowing one version of each package name to be installed. And the recently introduced container based replacements are and exercise in shooting twee twee birds with AAA...
  - lwf 10y ago
    Sure, which is why we have `libopenssl0.9.8`, `libopenssl1.0.0`, etc. In Debian at least, the policy is that package names for libraries should include something approximating the SONAME to allow for exactly this. Eventually you drop the old one, because you can no longer provide security support or for some other reason. But if there are compelling reasons otherwise, distributions can and do provide multiple ABIs for the same library.
    - digi_owl 10y ago
      And that just exasperates the problem.
      - Pxtl 10y ago
        At the very least shoulnd't they be providing and supporting multiple major versions on the same system? Major versions are, by definition, for breaking/incompatible changes, so having them being mutually exclusive creates impossible situations doesn't it?
        digi_owl 10y ago
        Well yes, but using the package name to distinguish them makes it harder for third parties to do packages. This is the complaint that has lead to the development of container based formats in the first place. Thing is that with sonames the actual files are distinguished anyways, so mangling the package names to get around collisions are just an artifact of an overly rigid package manager.
dunkelheit 10y ago
What I mean is that in addition to rules of the language I program in (say C or C++) there is another bunch of seemingly arbitrary rules that I must constantly be aware of to ensure nothing breaks. Rules like "don't reorder fields of structs" or "you can add new methods to a class as long as they are not virtual. and sometimes you can add virtual methods too as long as they end up at the end of vtable". How do I ensure that I conform to these rules apart from being disciplined about them? If everything compiles, am I good? No! If everything links, am I good? No! Depending on linker options and the nature of incompatibility the program using incompatible ABIs can blow up at runtime or just silently corrupt data. And don't get me started on the venerable "pimpl idiom". A page of boilerplate just to ensure the most basic thing. Hope that clarifies my short sentiment a bit :) I agree that once you grasp the rules following them is not that hard but it is just another bit of incidental complexity that we agreed to maintain.
- viraptor 10y ago
  I think most of the issues you listed are quite easy to check automatically. For example http://ispras.linuxbase.org/index.php/ABI_compliance_checker http://ispras.linuxbase.org/index.php/ABI_compliance_checker or anything in the "similar software" section. Of course it won't save you from business logic bugs (changing the meaning, but not size/location of a value), but "how do I ensure that I conform" is 90+% possible to verify automatically.
  - dunkelheit 10y ago
    Thanks, didn't know about it! Still, that's a 3-rd party tool and I still have to write my pimpls. I stand by my PITA sentiment :)
- rectang 10y ago
  C++ ABI compat rules really hurt when they prevent you from evolving a library in a sensible way. Say that you really need to add a method to a virtual class. Unless you planned ahead by adding placeholder methods, you're SOL. What a lousy design constraint! This problem is solvable at the language/runtime level by resolving vtable offsets at runtime, but that doesn't help the existing C++ userbase.

ascotan 10y ago

C/C++ shared libraries are very easy to work with and distribute. Unfortunately this is becoming a lost skill in the world of javascript. People inherently thing 'old = bad' and are doom to reinvent the same thing over and over.