3 ms·

(founder here). The interaction between high level language package managers and system binaries or compiled extensions is definitely on our roadmap. For exampl

(founder here). The interaction between high level language package managers and system binaries or compiled extensions is definitely on our roadmap. For example the ruby net-ssh gem relies on specific versions of openssl being compiled on the machine. Do you think this is particularly difficult in ML? Is your frustration mostly from needing compile all this non-python stuff across environments?

ZeroCool2u 3y ago
Yeah, you're right on the money. This is basically insane in ML, but _especially_ in corporate environments, think F500, where sudo is disabled in containers and experimentation (in terms of trying different package versions) is quite challenging due to security restrictions. Frankly, it's one of the reasons that we're seeing a lot of excitement around stuff written in Rust, but with Python bindings[1][2]. Cargo makes it so easy to build across various platforms and then just produce statically compiled platform specific python wheels. If PyTorch and TF had been written using Rust, any other other ultra high perf language with a tool-chain as reliable as cargo, a lot of folks lives would be easier. [1]: https://www.getdaft.io/ https://www.getdaft.io/ [2]: https://www.pola.rs/ https://www.pola.rs/
pabs3 3y ago
Hmm, the Debian package of ruby-net-ssh doesn't even depend on OpenSSL at all.
- stevepike 3y ago
  Interesting. Should it? I don't have much experience installing ruby gems with the OS package managers, I've always done it through the language-specific ones like bundler / gem. Here's a github issue showing the kind of things that come up when the versions are mismatched: https://github.com/net-ssh/net-ssh/issues/843 https://github.com/net-ssh/net-ssh/issues/843
  - pabs3 3y ago
    I was probably looking at an older or newer version to you, likely it switched from OpenSSL to pure-Ruby implementations of the crypto stuff.

ZeroCool2u 3y ago

Yeah, you're right on the money. This is basically insane in ML, but _especially_ in corporate environments, think F500, where sudo is disabled in containers and experimentation (in terms of trying different package versions) is quite challenging due to security restrictions. Frankly, it's one of the reasons that we're seeing a lot of excitement around stuff written in Rust, but with Python bindings[1][2]. Cargo makes it so easy to build across various platforms and then just produce statically compiled platform specific python wheels. If PyTorch and TF had been written using Rust, any other other ultra high perf language with a tool-chain as reliable as cargo, a lot of folks lives would be easier. [1]: https://www.getdaft.io/ https://www.getdaft.io/ [2]: https://www.pola.rs/ https://www.pola.rs/

pabs3 3y ago

Hmm, the Debian package of ruby-net-ssh doesn't even depend on OpenSSL at all.

stevepike 3y ago
Interesting. Should it? I don't have much experience installing ruby gems with the OS package managers, I've always done it through the language-specific ones like bundler / gem. Here's a github issue showing the kind of things that come up when the versions are mismatched: https://github.com/net-ssh/net-ssh/issues/843 https://github.com/net-ssh/net-ssh/issues/843
- pabs3 3y ago
  I was probably looking at an older or newer version to you, likely it switched from OpenSSL to pure-Ruby implementations of the crypto stuff.