4 ms·

I happen to know that pytorch is a pain to maintain in packaging systems. It has a complex build system, many non-python dependencies, and massive build times.

I happen to know that pytorch is a pain to maintain in packaging systems. It has a complex build system, many non-python dependencies, and massive build times. I don't know how this factors into the dissatisfaction with a C++ implementation, but I wouldn't be surprised if it were a factor. In other words, python binary wheels are harder to maintain than source-only python packages. And pytorch uses more than a few. I can't imagine Julia makes the problem much simpler. The main pain point is probably the lack of standard, multi-environment packaging solutions for natively compiled code. I don't know what it would take for this sort of pain point to improve significantly. Some standards around how C, C++, and Fortran projects are packaged would help. This would allow projects to build on top of existing natively compiled tech a lot better. Maybe the biggest reason those languages don't have the same "ecosystem" as python is utter lack of packaging standardization.

fock 5y ago
I would recommend you never try to compile tensorflow then. Honestly pytorch might take ages and be relatively complex. But 9/10 tries it builds on a recent Fedora, when you get a new release. tensorflow then ... I mean it's not me to judge that, as I don't understand why you would vendor a ton of C-libraries while depending on a single-file v0.0.3 python library...
adgjlsfhk1 5y ago
Julia actually makes this problem way better for 2 different reasons. the first is that you don't need nearly as much C/Fortran when you are working in a fast language. the second is that Julia has binarybuilder which is a really good system for delivering reproduceable binaries that can be distributed. To show how well this works, try adding the Cuda Julia library. it just works without any of the issues python has.
Sukera 5y ago
> The main pain point is probably the lack of standard, multi-environment packaging solutions for natively compiled code. Are you talking about something like BinaryBuilder.jl[1], which provides native binaries as julia-callable wrappers? -- [1] https://binarybuilder.org https://binarybuilder.org

fock 5y ago

I would recommend you never try to compile tensorflow then. Honestly pytorch might take ages and be relatively complex. But 9/10 tries it builds on a recent Fedora, when you get a new release. tensorflow then ... I mean it's not me to judge that, as I don't understand why you would vendor a ton of C-libraries while depending on a single-file v0.0.3 python library...

adgjlsfhk1 5y ago

Julia actually makes this problem way better for 2 different reasons. the first is that you don't need nearly as much C/Fortran when you are working in a fast language. the second is that Julia has binarybuilder which is a really good system for delivering reproduceable binaries that can be distributed. To show how well this works, try adding the Cuda Julia library. it just works without any of the issues python has.

Sukera 5y ago

> The main pain point is probably the lack of standard, multi-environment packaging solutions for natively compiled code. Are you talking about something like BinaryBuilder.jl[1], which provides native binaries as julia-callable wrappers? -- [1] https://binarybuilder.org https://binarybuilder.org