3 ms·

Concrete example: the Apache Arrow format is designed to allow passing arrays across languages and libraries without conversion; so many public functions/method

Concrete example: the Apache Arrow format is designed to allow passing arrays across languages and libraries without conversion; so many public functions/methods (and FFIs) will have Arrow arrays in their signature. However, the Rust Arrow library bumps its major version every few months because it changes methods and auxiliary types, without changing the memory layout of arrays. But this means you can't have two dependencies (eg. an ORC parser and a CSV serializer) that depend on different versions of Arrow if you want to pass arrays from one dependency's functions to the other's. And vendoring like you mention won't help, because the Rust compiler wouldn't recognize both vendored Arrow crates as compatible.

mgaunard 2y ago
Use a single version of the arrow library, and build everything against that one. I don't see where the problem is. I don't use Rust or third-party build or packaging systems -- I usually recommend that people don't do so either, but I understand that cargo is part of the appeal of Rust. I'd say just build whatever you need to do, this way you won't be limited by arbitrary restrictions others have decided.
- hinkley 2y ago
  Security patches say no, doesn’t work that way.
- progval 2y ago
  I can't always do that because dependency A depends dependency B that depends on arrow 52, and dependency C depends on arrow 53. And sure, I can soft-fork dependency B to add support for arrow 53, and dependency A to depend on my soft-fork of dependency B, but it quickly becomes messy. Especially when in parallel I send unrelated patches to A and/or B.
  - mgaunard 2y ago
    your problem is caring about what other people have decided. Surely metadata is irrelevant and can be changed/overridden. You onboard a version and make sure everything you need can build against it, patching software as needed.
hinkley 2y ago
Sounds like a team of misanthropes. How can you be that blind to DevEx?
- mgaunard 2y ago
  For the best developer experience, you need to write custom tools.
  - hinkley 2y ago
    Eh. Software is full of cliches and idioms. I shouldn’t be writing my own logging library and I sure as fuck shouldn’t be writing my own security library. Also don’t write your own billing system unless you’re a finance company.

mgaunard 2y ago

Use a single version of the arrow library, and build everything against that one. I don't see where the problem is. I don't use Rust or third-party build or packaging systems -- I usually recommend that people don't do so either, but I understand that cargo is part of the appeal of Rust. I'd say just build whatever you need to do, this way you won't be limited by arbitrary restrictions others have decided.

hinkley 2y ago
Security patches say no, doesn’t work that way.
progval 2y ago
I can't always do that because dependency A depends dependency B that depends on arrow 52, and dependency C depends on arrow 53. And sure, I can soft-fork dependency B to add support for arrow 53, and dependency A to depend on my soft-fork of dependency B, but it quickly becomes messy. Especially when in parallel I send unrelated patches to A and/or B.
- mgaunard 2y ago
  your problem is caring about what other people have decided. Surely metadata is irrelevant and can be changed/overridden. You onboard a version and make sure everything you need can build against it, patching software as needed.

hinkley 2y ago

Sounds like a team of misanthropes. How can you be that blind to DevEx?

mgaunard 2y ago
For the best developer experience, you need to write custom tools.
- hinkley 2y ago
  Eh. Software is full of cliches and idioms. I shouldn’t be writing my own logging library and I sure as fuck shouldn’t be writing my own security library. Also don’t write your own billing system unless you’re a finance company.