14 ms·

MLJ.jl: A Julia package for composable machine learning

234 points by amkkma 6y ago

teruakohatu 6y ago
Model composability is one of Julia's and MLJ's highlights. Library designers tend to take this into account throughout the ecosystem. It helps that Julia does not need to dropdown into a C or FORTRAN wrapper like Numpy. That being said, MLJ tends to wrap a lot of models from scikit-learn with a composable wrapper, meaning you end up having to manage Python dependencies and you lose Julia features such as multi-threading (Edit: when using Python models, multi-threading is supported when using Julia models). Long term I am sure MLJ will slowly replace scikit models but right now it can be little painful to use if the model you want is not implemented in Julia.
- snicker7 6y ago
  This is exacerbated by the fact that Julia's Pkg.jl does not yet support conditional/optional dependencies [0]. A lot of these meta packages tend to pull everything but the kitchen sink. [0]: https://github.com/JuliaLang/Pkg.jl/issues/1285 https://github.com/JuliaLang/Pkg.jl/issues/1285
- beforeolives 6y ago
  So instead of dropping down into C, it has Python as a depedency and wraps sklearn? Way to solve the two-language problem.
  - newswasboring 6y ago
    That's a genuinely mean comment towards a new library which is using existing code to bootstrap itself. Julia does solve the two language issue, but not in all situations. Like I'll never write a driver in julia. But this mean spiritedness towards anything that's not perfect is disheartening.
    - kescobo 6y ago
      >Julia does solve the two language issue, but not in all situations And not instantaneously. Developer time being finite, using what works from existing libraries while waiting for a pure-julia implementation seems like an excellent solution.
  - oscardssmith 6y ago
    The ability to call other languages is a strength. The need to do so is a weakness.
    - Jouvence 6y ago
      I disagree; consistency is the most important part of this. With Python, you know where you stand - performance comes from elsewhere. With Julia it might be internal, a C/Fortran library, or apparently other things now too.
      - DNF2 6y ago
        I beg your pardon? Knowing that Python is slow is a strength of Python? Then, if Python suddenly became fast, it would be a negative? Did I misunderstand something? Presumably, all fast languages are at a disadvantage, then.
        Jouvence 6y ago
        I never called it a strength; I simply don't care how fast Python is because I don't need to. Reasoning about what is going on under the hood with Python is just easier than Julia - if it needs to be fast, it's a fast external library being used. This is really a minor issue stemming from the relative maturity of the languages - if Julia becomes more established I would hope usage of external libraries which don't offer a performance advantage (ie everything besides C and Fortran) eventually gets replaced with native packages to preserve the sanity of the users.
        DNF2 6y ago
        So then that 'weakness' in python is not an issue to you personally, because you fluently drop down to a fast language anytime you need to, with no particular loss in productivity? Keep in mind though, that this could be a hurdle to those who are less multilingual. So even though it's not a weakness to everyone, it is to many.
        adgjlsfhk1 6y ago
  - notagoodidea 6y ago
    On the other side, we observe more and more inter-languages's ecosystem crossing from/to Python to leverage its gigantic set of libraries. Julia interface with a lot of other languages but it is a big trend for the last years. I think the two-language problem begins to be as we move forward a network of multi-languages calling each other instead of either having to rewrite everything from scratch either relying on a C/C++/Fortran/Rust lib and the C ABI to patch them. We may finish with a set of "meta"-langurust in C code nameages permitting the interaction of a large number of libraries and programming languages outside of their silo. Moreover Julia coupled with Pycall could "corrode" à la Rust for a more smooth migration to pure Julia.
  - Certhas 6y ago
    Julia has always had the ability to call Python code. For example you can use PyPlot/matplotlib as a plotting backend for Julia. There is no way to bootstrap a scientific language without leveraging existing ecosystems. But this is a starting point. Whenever someone find a sklearn model to slow, or misses the automatic parallelism that Julia brings, they can just implement the model in Julia, and bit by bit the non-Julia bits get swapped out where it brings a real benefit.
  - systems 6y ago
    I dont like Python, but the language have dominant libraries in many domains Julia hides Python, Julia good I understand that using Python feels wrong, because Python is not know for its performance, but still it seems like a Practical decision, rebuilding everything from scratch will be too much effort
  - dklend122 6y ago
    You hardly need python for models at this point. There are native julia implementations of linear regression, naive bayes, gradient boosting, random forest, PCA, T-SNE and more. The python interop is just a stopgap
  - ponow 6y ago
    Absa-bloody-lutely it's a way of solving the two language problem, because it provides a pathway from the current two-language setup (Python with C for the computationally intense portions) to a single language (Julia everywhere, because it can reach C speeds). In the short term, library writers can provide Julia wrappers around legacy code from other languages, but that needn't affect the library users. I think you can only make the snarky remark because you don't truly get the 2-language problem. The problem has zero to do with the complexity of dealing with dependencies, as if the hard problem of scientific computation languages is like a Linux distribution with too much bloat. No, no, no. It's the cognitive load to solve scientific problems, when at the end of the day performance really matters. We need a language that is as terse as mathematics where we can customize all the way down to a single array entry, without performance loss. Python itself will never have the necessary performance. To do truly new numerical stuff you cannot avoid the computational kernel details (e.g., PDEs or image processing). So you're forced, as a researcher, to become good enough to be effective at two languages, and you often have a complex data exchange interface between the two, and it sits right near the critical conceptual problems that you're trying to solve.
  - anothathrow975 6y ago
    Maybe this comment was too sarcastic but the core point stands. Julia’s main advantages for most ML and Data Science users are still theoretical. The package ecosystem is still way to immature. It’s not just MLJ. I was excited about Julia and made a good faith effort to port a work project to Flux and Zygkte and it was a disaster. There’s nothing wrong with passionate open source tinkerers working to improve the ecosystem, but don’t advertise it as ready for production when it’s not.
    - amkkma 6y ago
      When was this and what were the issues?
  - DNF2 6y ago
    The two-language problem is not concerned with calling out to or wrapping pre-existing libraries in a second language. It means having to write parts of your code in a second language, due to, for example, performance reasons.
deleted 6y ago
[deleted]
ampdepolymerase 6y ago
This looks very similar to https://github.com/IBM/AutoMLPipeline.jl https://github.com/IBM/AutoMLPipeline.jl
doctoboggan 6y ago
For those with experience with ML in Julia I’d love to get some advice. I have a little business that sells 3D printed jewelry (https://lulimjewelry.com https://lulimjewelry.com). My biggest seller is customers engraving their or their loved ones fingerprint on the ring. Most of those prints come in needing manual cleanup, which I can usually do in a few minutes. I’d love to train a ML algorithm to do this, and I’ve been building up the before and after pictures over time using my manually cleaned up customer fingerprint images. Can anyone give me suggestions or pointers on the sort of algorithm that may be best suited for this task? Just something to get me started down the correct path would be very helpful.
- tomrod 6y ago
  ML practitioner here. Step 1: write out the steps you do to manually clean up. Step 2: Depending on: (1) how many samples you have, (2) what is defined as "before" and "after", (3) what performance level is acceptable, there are many algorithms to choose from. Transform learning may be the target, reinforcement learning or CNN as well. It really depends on Step (1) and (2.1).
  - doctoboggan 6y ago
    For a given poor quality print I usually take it on my iPad and retrace either the whole print or just a portion. I usually just use the pencil tool in white and run it between the ridges of the print to give good separation and definition to the final engraving. Depending on the quality of the print (some of them are hardly more defined that a low res ink smudge) I may have to use a good amount of artistic liberties on where I think the pattern was going. Using my knowledge of what a print may look like I can usually do a decent job. Thats why I thought this might be a good task for a ML algorithm since I can show it what a bad print looks like and what a decent cleaned up version of that print should look like.
    - tomrod 6y ago
      So you're denoising by identifying where the patterns should be, but there is some smudge? If so, you may be able to skip ML altogether and use a deterministic software program to classify light versus dark areas (what you're doing with your pencil). ML might help with mapping smudges to the right shape/sizing. That said, if you have a ton of examples, then yeah an image recognition task might help.
Skyy93 6y ago
I do not understand why someone should use this? It is basically a wrapper of existing Libs and Frameworks. This framework does not solve any problem that scikitlearn and other existing frameworks have not already solved.
- sgt101 6y ago
  A good thing to read is https://joss.theoj.org/papers/10.21105/joss.02704 https://joss.theoj.org/papers/10.21105/joss.02704 The focus of this package is on the "plubming" of auto-ml solutions. The work of designing solutions based on multiple discovered models is rather unsupported in the current state of the art, and this package looks quite supportive of it. A big contribution (if it works in practice) is the idea of a scientific type describing what data is and how it should be mapped to algorithms - a systematic way of doing this is the underpinning of a process for model selection.
- dklend122 6y ago
  Check out the " Model composability" section. Also multithreading, custom differentiable loss functions, works with any abstract table type, and the list goes on
indeedmug 6y ago
There are some very cool features with MLJ. You can query models() and find all of the machine learning models that your data works with. This is great for easily plugging into various models to see what works. The problem I ran into is that it can be very unclear when things don't plugin exactly as planned. You get a typing error that looks like C++ template errors. Or your data doesn't work quite right with a model and needs some transforming but the documentation doesn't spell how to do with your situation. There aren't nearly as many StackOverflow questions on using Julia as there are Python ones so you can't just look up a very specific question and get answers.
- Certhas 6y ago
  Unfortunately this is a quite general issue with Julia at the moment. If things work it's magical, if they don't it's hard to see what subtle assumption of the packages used was violated. Julia and C++ Templates share a lot of properties, and some of the same pain points. It took C++ until C++20 to address this with Concepts. I hope it wont take quite as long for Julia.
Tarrosion 6y ago
I want to love MLJ and I _do_ love Julia, but holy wow is it hard to learn. I think three times now I've had a small dataset I wanted to do something simple like linear regression on, thought it'd be a good opportunity to learn MLJ, and ended up giving up when I was knee deep in inscrutable errors about scientific types and machines and unsupported models. If anyone has a good introduction to recommend which is clearer than the official docs, I'd definitely appreciate it.
- gugagore 6y ago
  Sorry, I don't have a good introduction to recommend, I just wanted to include a link if anyone is intrigued by "scientific types", which refers specifically to: https://github.com/alan-turing-institute/ScientificTypes.jl https://github.com/alan-turing-institute/ScientificTypes.jl
- indeedmug 6y ago
  The MLJ library has some example notebooks in the github repo somewhere. I used those notebooks to figure out the API of MLJ because there are unstated things in the documentation. But I agree, it's brutal to learn Julia because the documentation is so lacking for beginners and type errors can be very challenging to understand.
  - Tarrosion 6y ago
    FWIW I've found learning Julia (and using it daily) mostly overwhelmingly pleasant, and I perceive MLJ as an unfortunate outlier. Sounds like you had a different experience.
  - adgjlsfhk1 6y ago
    If you haven't tried out Julia recently, you might want to. 1.6 makes some pretty big improvements in stacktrace printing.

234 points by amkkma 6y ago

Model composability is one of Julia's and MLJ's highlights. Library designers tend to take this into account throughout the ecosystem. It helps that Julia does not need to dropdown into a C or FORTRAN wrapper like Numpy. That being said, MLJ tends to wrap a lot of models from scikit-learn with a composable wrapper, meaning you end up having to manage Python dependencies and you lose Julia features such as multi-threading (Edit: when using Python models, multi-threading is supported when using Julia models). Long term I am sure MLJ will slowly replace scikit models but right now it can be little painful to use if the model you want is not implemented in Julia.

snicker7 6y ago
This is exacerbated by the fact that Julia's Pkg.jl does not yet support conditional/optional dependencies [0]. A lot of these meta packages tend to pull everything but the kitchen sink. [0]: https://github.com/JuliaLang/Pkg.jl/issues/1285 https://github.com/JuliaLang/Pkg.jl/issues/1285
beforeolives 6y ago
So instead of dropping down into C, it has Python as a depedency and wraps sklearn? Way to solve the two-language problem.
- newswasboring 6y ago
  That's a genuinely mean comment towards a new library which is using existing code to bootstrap itself. Julia does solve the two language issue, but not in all situations. Like I'll never write a driver in julia. But this mean spiritedness towards anything that's not perfect is disheartening.
  - kescobo 6y ago
    >Julia does solve the two language issue, but not in all situations And not instantaneously. Developer time being finite, using what works from existing libraries while waiting for a pure-julia implementation seems like an excellent solution.
- oscardssmith 6y ago
  The ability to call other languages is a strength. The need to do so is a weakness.
  - Jouvence 6y ago
    I disagree; consistency is the most important part of this. With Python, you know where you stand - performance comes from elsewhere. With Julia it might be internal, a C/Fortran library, or apparently other things now too.
    - DNF2 6y ago
      I beg your pardon? Knowing that Python is slow is a strength of Python? Then, if Python suddenly became fast, it would be a negative? Did I misunderstand something? Presumably, all fast languages are at a disadvantage, then.
      - Jouvence 6y ago
        I never called it a strength; I simply don't care how fast Python is because I don't need to. Reasoning about what is going on under the hood with Python is just easier than Julia - if it needs to be fast, it's a fast external library being used. This is really a minor issue stemming from the relative maturity of the languages - if Julia becomes more established I would hope usage of external libraries which don't offer a performance advantage (ie everything besides C and Fortran) eventually gets replaced with native packages to preserve the sanity of the users.
        DNF2 6y ago
        So then that 'weakness' in python is not an issue to you personally, because you fluently drop down to a fast language anytime you need to, with no particular loss in productivity? Keep in mind though, that this could be a hurdle to those who are less multilingual. So even though it's not a weakness to everyone, it is to many.
        adgjlsfhk1 6y ago
- notagoodidea 6y ago
  On the other side, we observe more and more inter-languages's ecosystem crossing from/to Python to leverage its gigantic set of libraries. Julia interface with a lot of other languages but it is a big trend for the last years. I think the two-language problem begins to be as we move forward a network of multi-languages calling each other instead of either having to rewrite everything from scratch either relying on a C/C++/Fortran/Rust lib and the C ABI to patch them. We may finish with a set of "meta"-langurust in C code nameages permitting the interaction of a large number of libraries and programming languages outside of their silo. Moreover Julia coupled with Pycall could "corrode" à la Rust for a more smooth migration to pure Julia.
- Certhas 6y ago
  Julia has always had the ability to call Python code. For example you can use PyPlot/matplotlib as a plotting backend for Julia. There is no way to bootstrap a scientific language without leveraging existing ecosystems. But this is a starting point. Whenever someone find a sklearn model to slow, or misses the automatic parallelism that Julia brings, they can just implement the model in Julia, and bit by bit the non-Julia bits get swapped out where it brings a real benefit.
- systems 6y ago
  I dont like Python, but the language have dominant libraries in many domains Julia hides Python, Julia good I understand that using Python feels wrong, because Python is not know for its performance, but still it seems like a Practical decision, rebuilding everything from scratch will be too much effort
- dklend122 6y ago
  You hardly need python for models at this point. There are native julia implementations of linear regression, naive bayes, gradient boosting, random forest, PCA, T-SNE and more. The python interop is just a stopgap
- ponow 6y ago
  Absa-bloody-lutely it's a way of solving the two language problem, because it provides a pathway from the current two-language setup (Python with C for the computationally intense portions) to a single language (Julia everywhere, because it can reach C speeds). In the short term, library writers can provide Julia wrappers around legacy code from other languages, but that needn't affect the library users. I think you can only make the snarky remark because you don't truly get the 2-language problem. The problem has zero to do with the complexity of dealing with dependencies, as if the hard problem of scientific computation languages is like a Linux distribution with too much bloat. No, no, no. It's the cognitive load to solve scientific problems, when at the end of the day performance really matters. We need a language that is as terse as mathematics where we can customize all the way down to a single array entry, without performance loss. Python itself will never have the necessary performance. To do truly new numerical stuff you cannot avoid the computational kernel details (e.g., PDEs or image processing). So you're forced, as a researcher, to become good enough to be effective at two languages, and you often have a complex data exchange interface between the two, and it sits right near the critical conceptual problems that you're trying to solve.
- anothathrow975 6y ago
  Maybe this comment was too sarcastic but the core point stands. Julia’s main advantages for most ML and Data Science users are still theoretical. The package ecosystem is still way to immature. It’s not just MLJ. I was excited about Julia and made a good faith effort to port a work project to Flux and Zygkte and it was a disaster. There’s nothing wrong with passionate open source tinkerers working to improve the ecosystem, but don’t advertise it as ready for production when it’s not.
  - amkkma 6y ago
    When was this and what were the issues?
- DNF2 6y ago
  The two-language problem is not concerned with calling out to or wrapping pre-existing libraries in a second language. It means having to write parts of your code in a second language, due to, for example, performance reasons.

deleted 6y ago

[deleted]

ampdepolymerase 6y ago

This looks very similar to https://github.com/IBM/AutoMLPipeline.jl https://github.com/IBM/AutoMLPipeline.jl

doctoboggan 6y ago

For those with experience with ML in Julia I’d love to get some advice. I have a little business that sells 3D printed jewelry (https://lulimjewelry.com https://lulimjewelry.com). My biggest seller is customers engraving their or their loved ones fingerprint on the ring. Most of those prints come in needing manual cleanup, which I can usually do in a few minutes. I’d love to train a ML algorithm to do this, and I’ve been building up the before and after pictures over time using my manually cleaned up customer fingerprint images. Can anyone give me suggestions or pointers on the sort of algorithm that may be best suited for this task? Just something to get me started down the correct path would be very helpful.

tomrod 6y ago
ML practitioner here. Step 1: write out the steps you do to manually clean up. Step 2: Depending on: (1) how many samples you have, (2) what is defined as "before" and "after", (3) what performance level is acceptable, there are many algorithms to choose from. Transform learning may be the target, reinforcement learning or CNN as well. It really depends on Step (1) and (2.1).
- doctoboggan 6y ago
  For a given poor quality print I usually take it on my iPad and retrace either the whole print or just a portion. I usually just use the pencil tool in white and run it between the ridges of the print to give good separation and definition to the final engraving. Depending on the quality of the print (some of them are hardly more defined that a low res ink smudge) I may have to use a good amount of artistic liberties on where I think the pattern was going. Using my knowledge of what a print may look like I can usually do a decent job. Thats why I thought this might be a good task for a ML algorithm since I can show it what a bad print looks like and what a decent cleaned up version of that print should look like.
  - tomrod 6y ago
    So you're denoising by identifying where the patterns should be, but there is some smudge? If so, you may be able to skip ML altogether and use a deterministic software program to classify light versus dark areas (what you're doing with your pencil). ML might help with mapping smudges to the right shape/sizing. That said, if you have a ton of examples, then yeah an image recognition task might help.

Skyy93 6y ago

I do not understand why someone should use this? It is basically a wrapper of existing Libs and Frameworks. This framework does not solve any problem that scikitlearn and other existing frameworks have not already solved.

sgt101 6y ago
A good thing to read is https://joss.theoj.org/papers/10.21105/joss.02704 https://joss.theoj.org/papers/10.21105/joss.02704 The focus of this package is on the "plubming" of auto-ml solutions. The work of designing solutions based on multiple discovered models is rather unsupported in the current state of the art, and this package looks quite supportive of it. A big contribution (if it works in practice) is the idea of a scientific type describing what data is and how it should be mapped to algorithms - a systematic way of doing this is the underpinning of a process for model selection.
dklend122 6y ago
Check out the " Model composability" section. Also multithreading, custom differentiable loss functions, works with any abstract table type, and the list goes on

indeedmug 6y ago

There are some very cool features with MLJ. You can query models() and find all of the machine learning models that your data works with. This is great for easily plugging into various models to see what works. The problem I ran into is that it can be very unclear when things don't plugin exactly as planned. You get a typing error that looks like C++ template errors. Or your data doesn't work quite right with a model and needs some transforming but the documentation doesn't spell how to do with your situation. There aren't nearly as many StackOverflow questions on using Julia as there are Python ones so you can't just look up a very specific question and get answers.

Certhas 6y ago
Unfortunately this is a quite general issue with Julia at the moment. If things work it's magical, if they don't it's hard to see what subtle assumption of the packages used was violated. Julia and C++ Templates share a lot of properties, and some of the same pain points. It took C++ until C++20 to address this with Concepts. I hope it wont take quite as long for Julia.

Tarrosion 6y ago

I want to love MLJ and I _do_ love Julia, but holy wow is it hard to learn. I think three times now I've had a small dataset I wanted to do something simple like linear regression on, thought it'd be a good opportunity to learn MLJ, and ended up giving up when I was knee deep in inscrutable errors about scientific types and machines and unsupported models. If anyone has a good introduction to recommend which is clearer than the official docs, I'd definitely appreciate it.

gugagore 6y ago
Sorry, I don't have a good introduction to recommend, I just wanted to include a link if anyone is intrigued by "scientific types", which refers specifically to: https://github.com/alan-turing-institute/ScientificTypes.jl https://github.com/alan-turing-institute/ScientificTypes.jl
indeedmug 6y ago
The MLJ library has some example notebooks in the github repo somewhere. I used those notebooks to figure out the API of MLJ because there are unstated things in the documentation. But I agree, it's brutal to learn Julia because the documentation is so lacking for beginners and type errors can be very challenging to understand.
- Tarrosion 6y ago
  FWIW I've found learning Julia (and using it daily) mostly overwhelmingly pleasant, and I perceive MLJ as an unfortunate outlier. Sounds like you had a different experience.
- adgjlsfhk1 6y ago
  If you haven't tried out Julia recently, you might want to. 1.6 makes some pretty big improvements in stacktrace printing.