11 ms·

I love all languages in the ML/Haskell tradition, but I think Julia would have been a better fit because it's dynamic yet efficient, and because it has a really

by nextos 6y ago

I love all languages in the ML/Haskell tradition, but I think Julia would have been a better fit because it's dynamic yet efficient, and because it has a really decent probability/statistics/ML ecosystem already. Long term, I think it's the best replacement we have for Python in the ML world. Python has exceptional libraries but, as a language, it's a bit dated on several fronts. This has an impact on library design. In Python, ML libraries are huge monoliths and they depend on a lot of code written in other languages. They are really hard to understand or modify. In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks. Same applies to many other things. It's small and beautiful, but it needs more manpower to compete against Python.

Karrot_Kream 6y ago
> In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks. I was taken aback when looking at Turing for Bayesian modelling that the distributions were just the standard distributions found in the Distributions package! In Python, every Bayesian framework has its own implementation of everything from distributions to log probabilities, but it all composes in Julia. > It's small and beautiful, but it needs more manpower to compete against Python. Agreed. Docs of major projects are still incomplete or non-existent, there's a lot of projects that have been nigh abandoned (cough naive bayes cough), and the composability of Julia coupled without strong leadership leads to an ecosystem of overlapping functionality. Still, I hope by leveraging PyCall, Julia can overcome some of these issues.
- renerthr 6y ago
  > but it all composes in Julia. You mean that, in Julia, we the users have to "compose" our own implementations of models (e.g. log probabilities), as opposed to using the already-made ones in Python?
  - cwyers 6y ago
    More like... most of the scientific Python stack has settled on a third party library, NumPy, to provide support for numerical computing. That includes providing data types like arrays. Tensorflow doesn't use NumPy's data types, so if you want to use Tensorflow Probability, you need to convert everything you've done in NumPy (or things that are built on top of NumPy, like Pandas) to TensorFlow, and use TensorFlow's functions/methods to manipulate them. (The same is true if you use something that's built on top of Stan or JAGS or BUGS or something else.) In Julia's Turing.jl, everything is built around data structures that are first-class parts of Julia, so there's no need to have special Turing.jl versions of, say, probability distributions.
    - The_rationalist 6y ago
      https://www.tensorflow.org/guide/tf_numpy https://www.tensorflow.org/guide/tf_numpy
      - dd82 6y ago
        yes, and if you go to that link, you can see its experimental, for TF 2.4 _and_ the notebook on github was created Aug 3, 2020. So that is relatively new and most likely has a ways to go before further adoption. Its good that option exists, but doesn't seem very backwards compatible. That doesn't negate the points made in the parent.
        The_rationalist 6y ago
        It will mature before the Julia ecosystem mature so the value proposition of Julia is out weighted by it's lack of ecosystem. Had they transparant interoperability with Python like Kotlin is for Java, I would consider it. They could at least rewrite their vm to use the truffle framework from graalvm so that they would become interoperable with Python on graal which in a few years will mature.
        amkkma 6y ago
swagonomixxx 6y ago
> Python has exceptional libraries but, as a language, it's a bit dated on several fronts. I've been using Python for ML for the last 3 years and I've never felt this way. It might be that I'm not all about the hip new languages, but I don't really see the benefit of making Python more ML/Haskell-ish. The ML use case for Python is roughly as follows: you design, train, and evaluate models. Then, if something is decent enough to use in production, you switch over your application to load and use that instead. I don't really see where Haskell or any language from the ML family can improve in that process. Sure, the code that you used to implement your model may improve slightly, but I don't see that code improving significantly. The fruit of your labor is usually a protobuf file (encoding the TensorFlow graph) or whatever your framework uses to encode the model you built. The code actually surrounding it is very minimal for most use cases. > In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks. There's nothing stopping you from composing things in Python. But it's simply not a goal of 99% of ML libraries to be composable with other libraries. You're probably never gonna run TensorFlow and PyTorch in the same application (trust me, I've tried, it was a nightmare) and I don't see why you would compose a TensorFlow model with a PyTorch model without incurring tons of overhead in your application around gluing these two things.
- amkkma 6y ago
  >there's nothing stopping you from composing things in Python. In Julia you can compose a custom distribution, with a bayesian model with an ODE with a neural network with unit number types with custom Julia written CUDA kernels and multithreading. Edit: That are not designed specifically to work with each other Can python even hope to do a fraction of that, still be fast and differentiate through everything?
  - swagonomixxx 6y ago
    I'm fairly certain everything you said is possible except for custom CUDA kernels in pure Python. You'd have to write the kernel in C++ and use it in your TensorFlow/PyTorch code. [0][1] [0]: https://www.tensorflow.org/guide/create_op https://www.tensorflow.org/guide/create_op [1]: https://pytorch.org/tutorials/advanced/cpp_extension.html https://pytorch.org/tutorials/advanced/cpp_extension.html
ZeroCool2u 6y ago
function julia_blues(bummers...) I largely agree, Julia is such a cool language and had so much potential. It definitely surprised me when they went with Swift instead, but realizing that Chris Lattner worked at Google at the time explained a lot. Unfortunately, every time I try to get into Julia, it just feels awkward coming from Python and a bit like stepping back in time. The stupidest, (stupidest in the sense that I really wish they didn't bother me, because they're silly things!), things that bother me are: 1. The 'end' keyword. It's everywhere! Loops, if-else statements, function bodies. I mean I understand why it's helpful for parsing and providing more info to the compiler, but honestly, I would've rather we just stuck with curly braces '{}'! At least it's fewer characters to type and it just feels less crowded on screen while reading. It feels like such a petty complaint, but honestly, it feels like I'm writing Pascal or Matlab all over again. Which leads me to my second point. 2. The default choice of 1 based indexing. I'm not going to go into it, because plenty of people before me have beat this dead horse already[1][2][3], but I can't help but be saddened by this choice and its implications. It's important to acknowledge the fact that Julia started as a competitor to Matlab and Octav, so it makes sense from that perspective. However, it could have been a much more general purpose language, with huge performance benefits over popular interpreted languages like Python, JS, and Ruby. It could have been a unifying force in the scientific computing community, bridging the gap between R and Python users with a greenfield approach that would have been a force to be reckoned with. Instead, rightly or not, it's viewed largely as 'just' a matlab replacement. Now, regardless of whether 1 or 0 based indexing is truly 'better' or the 'end' keyword is no big deal, the reality is that there's a huge demographic of potential users that won't buy into Julia, because it doesn't quite feel as ergonomic as Python/JS/Ruby and won't take it seriously as a general purpose language, because it looks/feels like Matlab and they only use 'real' programming languages. Again, I'm not saying this is right, but it is the reality we're faced with and it just feels like a huge missed opportunity and bums me out. end 1. https://github.com/JuliaLang/julia/pull/16260#issuecomment-217717018 https://github.com/JuliaLang/julia/pull/16260#issuecomment-2... 2. https://groups.google.com/g/julia-dev/c/tNN72FnYbYQ?pli=1 https://groups.google.com/g/julia-dev/c/tNN72FnYbYQ?pli=1 3. https://github.com/julialang/julia/issues/558 https://github.com/julialang/julia/issues/558
- skohan 6y ago
spamizbad 6y ago
Julia gets compared a lot to Python but truthfully it strikes me as more like a replacement for FORTRAN, MATLAB, and R. The ergonomics of the language seem good but documentation is poor and as a community it's narrowly scoped into the ML/numerical processing world. If all you do is ML/numerical processing, against data that's already been cleaned up, I bet it's really great tho.
- socialdemocrat 6y ago
  The comparison is natural because Python is a general purpose language which happens to be used for data science, the same can be said about Julia. The others languages mentioned are not really that great for general purpose programming. But I agree with you that in the short term it is a replacement for Fortran, Matlab and R. But a new language has to start somewhere. Attacking the niche filled by these language is natural as these are old, dated and ripe for disruption. The key is to gain some critical momentum and user base. That will carry the language to other domains.

by nextos 6y ago

Karrot_Kream 6y ago

> In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks. I was taken aback when looking at Turing for Bayesian modelling that the distributions were just the standard distributions found in the Distributions package! In Python, every Bayesian framework has its own implementation of everything from distributions to log probabilities, but it all composes in Julia. > It's small and beautiful, but it needs more manpower to compete against Python. Agreed. Docs of major projects are still incomplete or non-existent, there's a lot of projects that have been nigh abandoned (cough naive bayes cough), and the composability of Julia coupled without strong leadership leads to an ecosystem of overlapping functionality. Still, I hope by leveraging PyCall, Julia can overcome some of these issues.

renerthr 6y ago
> but it all composes in Julia. You mean that, in Julia, we the users have to "compose" our own implementations of models (e.g. log probabilities), as opposed to using the already-made ones in Python?
- cwyers 6y ago
  More like... most of the scientific Python stack has settled on a third party library, NumPy, to provide support for numerical computing. That includes providing data types like arrays. Tensorflow doesn't use NumPy's data types, so if you want to use Tensorflow Probability, you need to convert everything you've done in NumPy (or things that are built on top of NumPy, like Pandas) to TensorFlow, and use TensorFlow's functions/methods to manipulate them. (The same is true if you use something that's built on top of Stan or JAGS or BUGS or something else.) In Julia's Turing.jl, everything is built around data structures that are first-class parts of Julia, so there's no need to have special Turing.jl versions of, say, probability distributions.
  - The_rationalist 6y ago
    https://www.tensorflow.org/guide/tf_numpy https://www.tensorflow.org/guide/tf_numpy
    - dd82 6y ago
      yes, and if you go to that link, you can see its experimental, for TF 2.4 _and_ the notebook on github was created Aug 3, 2020. So that is relatively new and most likely has a ways to go before further adoption. Its good that option exists, but doesn't seem very backwards compatible. That doesn't negate the points made in the parent.
      - The_rationalist 6y ago
        It will mature before the Julia ecosystem mature so the value proposition of Julia is out weighted by it's lack of ecosystem. Had they transparant interoperability with Python like Kotlin is for Java, I would consider it. They could at least rewrite their vm to use the truffle framework from graalvm so that they would become interoperable with Python on graal which in a few years will mature.
        amkkma 6y ago

swagonomixxx 6y ago

> Python has exceptional libraries but, as a language, it's a bit dated on several fronts. I've been using Python for ML for the last 3 years and I've never felt this way. It might be that I'm not all about the hip new languages, but I don't really see the benefit of making Python more ML/Haskell-ish. The ML use case for Python is roughly as follows: you design, train, and evaluate models. Then, if something is decent enough to use in production, you switch over your application to load and use that instead. I don't really see where Haskell or any language from the ML family can improve in that process. Sure, the code that you used to implement your model may improve slightly, but I don't see that code improving significantly. The fruit of your labor is usually a protobuf file (encoding the TensorFlow graph) or whatever your framework uses to encode the model you built. The code actually surrounding it is very minimal for most use cases. > In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks. There's nothing stopping you from composing things in Python. But it's simply not a goal of 99% of ML libraries to be composable with other libraries. You're probably never gonna run TensorFlow and PyTorch in the same application (trust me, I've tried, it was a nightmare) and I don't see why you would compose a TensorFlow model with a PyTorch model without incurring tons of overhead in your application around gluing these two things.

amkkma 6y ago
>there's nothing stopping you from composing things in Python. In Julia you can compose a custom distribution, with a bayesian model with an ODE with a neural network with unit number types with custom Julia written CUDA kernels and multithreading. Edit: That are not designed specifically to work with each other Can python even hope to do a fraction of that, still be fast and differentiate through everything?
- swagonomixxx 6y ago
  I'm fairly certain everything you said is possible except for custom CUDA kernels in pure Python. You'd have to write the kernel in C++ and use it in your TensorFlow/PyTorch code. [0][1] [0]: https://www.tensorflow.org/guide/create_op https://www.tensorflow.org/guide/create_op [1]: https://pytorch.org/tutorials/advanced/cpp_extension.html https://pytorch.org/tutorials/advanced/cpp_extension.html

ZeroCool2u 6y ago

function julia_blues(bummers...) I largely agree, Julia is such a cool language and had so much potential. It definitely surprised me when they went with Swift instead, but realizing that Chris Lattner worked at Google at the time explained a lot. Unfortunately, every time I try to get into Julia, it just feels awkward coming from Python and a bit like stepping back in time. The stupidest, (stupidest in the sense that I really wish they didn't bother me, because they're silly things!), things that bother me are: 1. The 'end' keyword. It's everywhere! Loops, if-else statements, function bodies. I mean I understand why it's helpful for parsing and providing more info to the compiler, but honestly, I would've rather we just stuck with curly braces '{}'! At least it's fewer characters to type and it just feels less crowded on screen while reading. It feels like such a petty complaint, but honestly, it feels like I'm writing Pascal or Matlab all over again. Which leads me to my second point. 2. The default choice of 1 based indexing. I'm not going to go into it, because plenty of people before me have beat this dead horse already[1][2][3], but I can't help but be saddened by this choice and its implications. It's important to acknowledge the fact that Julia started as a competitor to Matlab and Octav, so it makes sense from that perspective. However, it could have been a much more general purpose language, with huge performance benefits over popular interpreted languages like Python, JS, and Ruby. It could have been a unifying force in the scientific computing community, bridging the gap between R and Python users with a greenfield approach that would have been a force to be reckoned with. Instead, rightly or not, it's viewed largely as 'just' a matlab replacement. Now, regardless of whether 1 or 0 based indexing is truly 'better' or the 'end' keyword is no big deal, the reality is that there's a huge demographic of potential users that won't buy into Julia, because it doesn't quite feel as ergonomic as Python/JS/Ruby and won't take it seriously as a general purpose language, because it looks/feels like Matlab and they only use 'real' programming languages. Again, I'm not saying this is right, but it is the reality we're faced with and it just feels like a huge missed opportunity and bums me out. end 1. https://github.com/JuliaLang/julia/pull/16260#issuecomment-217717018 https://github.com/JuliaLang/julia/pull/16260#issuecomment-2... 2. https://groups.google.com/g/julia-dev/c/tNN72FnYbYQ?pli=1 https://groups.google.com/g/julia-dev/c/tNN72FnYbYQ?pli=1 3. https://github.com/julialang/julia/issues/558 https://github.com/julialang/julia/issues/558

skohan 6y ago

spamizbad 6y ago

Julia gets compared a lot to Python but truthfully it strikes me as more like a replacement for FORTRAN, MATLAB, and R. The ergonomics of the language seem good but documentation is poor and as a community it's narrowly scoped into the ML/numerical processing world. If all you do is ML/numerical processing, against data that's already been cleaned up, I bet it's really great tho.

socialdemocrat 6y ago
The comparison is natural because Python is a general purpose language which happens to be used for data science, the same can be said about Julia. The others languages mentioned are not really that great for general purpose programming. But I agree with you that in the short term it is a replacement for Fortran, Matlab and R. But a new language has to start somewhere. Attacking the niche filled by these language is natural as these are old, dated and ripe for disruption. The key is to gain some critical momentum and user base. That will carry the language to other domains.