11 ms·

Scikit-Learn Version 1.0

260 points by m3at 5y ago

zibzab 5y ago
Is anyone using scikit for NN? Why/why not?
- armcat 5y ago
  NN as in "neural network", or NN as in "nearest neighbour" algorithm? No to the former, yes to the latter. The reason for a "no" to neural networks - in my case I've only ever implemented neural networks with many layers, and typically using kernels, pooling mechanisms, etc, and since scikit-learn doesn't have GPU support, I opt for frameworks that do (PyTorch, TensorFlow). However, if you're only building fully-connected neural nets (MLPs), with just a few layers, you don't need GPU support since any benefits of having parallel processing are offset by shuffling data between CPU and GPU. So in that case, scikit-learn would probably work quite well, although I never tested this myself.
  - woko 5y ago
    GPU can be useful for Nearest Neighbour as well. In case you have access to a GPU, I would strongly recommend Facebook's FAISS [1,2]. For everything else, sklearn is amazing. [1] https://faiss.ai/ https://faiss.ai/ [2] https://github.com/facebookresearch/faiss https://github.com/facebookresearch/faiss
    - armcat 5y ago
      Faiss looks very nice, thanks for the tip!
  - zibzab 5y ago
    But ignoring acceleration, could one use scikit instead of tesnorflow? I mean, ignoring everything else, scikit has a much friendlier API.
- jptech 5y ago
  You should be asking if anyone is using it for ML, not NN.
  - detaro 5y ago
    why?
- FlyingSaucer 5y ago
  I have used the MLP classifier[1] before. It's very simple to use (like most of sklearn's models). Worked well for standard and reasonably small classification model, but lacks some features for it to be a flexible way of using NNs: - No saving checkpoints (can be crucial for large models who need alot of compute and time) - No way to assign different activation functions to different layers - No complex nodes like LSTM, GRU - No way to implement complex architectures like transformers, encoders etc I also do not know if its even possible to use CUDA or any GPU with it. [1] : https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html#sklearn.neural_network.MLPClassifier https://scikit-learn.org/stable/modules/generated/sklearn.ne...
  - kuu 5y ago
    I would say the same as you. As long as you only need a simple model, yes, the MLP is good enough, but forget about making any DeepLearning stuff. And AFAIK, there isn't GPU support, CPU performance is poor compared to GPU execution.
- westurner 5y ago
  There are scikit-learn (sklearn) API-compatible wrappers for e.g. PyTorch and TensorFlow. Skorch: https://github.com/skorch-dev/skorch https://github.com/skorch-dev/skorch tf.keras.wrappers.scikit_learn: https://www.tensorflow.org/api_docs/python/tf/keras/wrappers/scikit_learn https://www.tensorflow.org/api_docs/python/tf/keras/wrappers... AFAIU, there are not Yellowbrick visualizers for PyTorch or TensorFlow; though PyTorch abd TensorFlow work with TensorBoard for visualizing CFG execution. > Many machine learning libraries implement the scikit-learn `estimator API` to easily integrate alternative optimization or decision methods into a data science workflow. Because of this, it seems like it should be simple to drop in a non-scikit-learn estimator into a Yellowbrick visualizer, and in principle, it is. However, the reality is a bit more complicated. > Yellowbrick visualizers often utilize more than just the method interface of estimators (e.g. `fit()` and `predict()`), relying on the learned attributes (object properties with a single underscore suffix, e.g. `coef_`). The issue is that when a third-party estimator does not expose these attributes, truly gnarly exceptions and tracebacks occur. Yellowbrick is meant to aid machine learning diagnostics reasoning, therefore instead of just allowing drop-in functionality that may cause confusion, we’ve created a wrapper functionality that is a bit kinder with it’s messaging. Looks like there are Yellowbrick wrappers for XGBoost, CatBoost, CuML, and Spark MLib; but not for NNs yet. https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html#tested-libraries https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html... From the RAPIDS.ai CuML team: https://docs.rapids.ai/api/cuml/stable/ https://docs.rapids.ai/api/cuml/stable/ : > cuML is a suite of fast, GPU-accelerated machine learning algorithms designed for data science and analytical tasks. Our API mirrors Sklearn’s, and we provide practitioners with the easy fit-predict-transform paradigm without ever having to program on a GPU. > As data gets larger, algorithms running on a CPU becomes slow and cumbersome. RAPIDS provides users a streamlined approach where data is intially loaded in the GPU, and compute tasks can be performed on it directly. CuML is not an NN library; but there are likely performance optimizations from CuDF and CuML that would accelerate performance of NNs as well. Dask ML works with models with sklearn interfaces, XGBoost, LightGBM, PyTorch, and TensorFlow: https://ml.dask.org/ https://ml.dask.org/ : > Scikit-Learn API > In all cases Dask-ML endeavors to provide a single unified interface around the familiar NumPy, Pandas, and Scikit-Learn APIs. Users familiar with Scikit-Learn should feel at home with Dask-ML. dask-labextension for JupyterLab helps to visualize Dask ML CFGs which call predictors and classifiers with sklearn interfaces: https://github.com/dask/dask-labextension https://github.com/dask/dask-labextension
- woko 5y ago
  See the FAQ: https://scikit-learn.org/dev/faq.html#will-you-add-gpu-support https://scikit-learn.org/dev/faq.html#will-you-add-gpu-suppo...
  - deleted 5y ago
    [deleted]
NeutralForest 5y ago
Excellent library with stellar documentation, I hope it'll live on for a long time.
- jgilias 5y ago
  Yes, glad they've decided it's finally out of the "don't use it, it's experimental" phase and gotten off the 0ver.org wall of shame!
- Uberphallus 5y ago
  Really, for any other ML library the best documentation is how-tos spread through the web, but scikit-learn leaves very little room for that kind of content.
- sveme 5y ago
  Best documented library. It even provides examples, guidance and best practices in the documentation. Have rarely learned so much as when I went through the sci-kit documentation. Absolute delight.
  - mistrial9 5y ago
    you mean the 4000 page cookbook thing?
monkeybutton 5y ago
Scikit-Learn is great, and, reading the documentation for other 3rd party ML packages and seeing the words "Scikit-learn API" is even better.
lysecret 5y ago
Excellent library for train_test_split. Jokes aside. This next to Numpy, Pandas Jupyter and Matplotlib + the DL libraries are the reason Python is the powerhouse it is for Data Science.
- disgruntledphd2 5y ago
  I'm with you on sklearn, the DL libraries and Numpy, but Pandas and Matplotlib are poor, poor relations of the tools available in the R ecosystem (dplyr/ggplot etc).
  - pc86 5y ago
    If you're doing data science aren't sklearn, DL, and numpy getting you 90% of the way there anyway? Even if R has better "versions" of pandas/matplotlib (not conceding that point) it's not exactly central to the job of data science.
    - civilized 5y ago
      As a working data scientist I'd say it's completely the opposite: a good tabular data manipulation package is the single most valuable tool in my tool box. And R's packages (either data.table or dplyr) are definitely way better than pandas. There's no comparison. I would be hard-pressed to find a working data scientist whose definition of data science is "that thing you do with sklearn, Deep Learning and Numpy".
      - jstx1 5y ago
        > "Data science is that thing where you do sklearn, Deep Learning and Numpy" is not a working data scientist's perspective. It could be. It's such a broad job title and it looks so different across different companies and teams that the main tool for one data scientist might be something that another data scientist never has to touch. Different data science jobs prioritise different tools, that's all.
        5y ago
infimum 5y ago
scikit-learn (next to numpy) is the one library I use in every single project at work. Every time I consider switching away from python I am faced with the fact that I'd lose access to this workhorse of a library. Of course it's not all sunshine and rainbows - I had my fair share of rummaging through its internals - but its API design is a de-facto standard for a reason. My only recurring gripe is that the serialization story (basically just pickling everything) is not optimal.
- CapmCrackaWaka 5y ago
  I recently ran into this issue as well. Serialization of sklearn random forests results in absolutely massive files. I had to switch to lightgbm, which is 100x faster to load from a save file and about 20x smaller.
- zeec123 5y ago
  There is so much wrong with the api design of sklearn (how can one think "predict_proba" is a good function name?). I can understand this, since most of it was probably written by PhD students without the time and expertise to come up with a proper api; many of them without a CS background.[1] [1] https://www.reddit.com/r/haskell/comments/7brsuu/machine_learning_in_functional_programming/dppck1x?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/haskell/comments/7brsuu/machine_lea...
  - mrtranscendence 5y ago
    I didn't want to bag on sklearn (I've already bagged on pandas enough here), but for what it's worth I agree with you. It's, ahh, not the API I would've come up with. It's what everybody has standardized on, though, and maybe there's some value in that.
  - kzrdude 5y ago
    These seem like minor gripes (reading your link) - and I don't even agree with them, seems like an ok use of mutable state (otherwise a separate object would be needed for hyperparameter state?). Maybe my expectations are low, but they way sklearn unifies the API across different estimators all across the library - that's already way above what you can expect - especially if you consider it to be "written by a bunch of phd students".
lr1970 5y ago
Just to clarify, scikit-learn 1.0 has not been released yet. The latest tag in the github repo is 1.0.rc2 https://github.com/scikit-learn/scikit-learn/releases/tag/1.0.rc2 https://github.com/scikit-learn/scikit-learn/releases/tag/1....
conor_f 5y ago
https://0ver.org/ https://0ver.org/ will need an update!
lr1970 5y ago
Early on, pandas made some unfortunate design decisions that are still biting hard. For example, the choice of datetime (pandas.Timestamp) represented by a 64-bit int with a fixed nanosecond resolution. This choice gives dynamic range of +- 292 years around 1970-01-01 (the epoch). This range is too small to represent the works of William Shakespeare, never mind human history. Using pandas in these areas becomes a royal pain in the neck, for one constantly needs to work around pandas datetime limitations. OTOH, in numpy one can choose time resolution units (anything from attosecond to a year) tailoring time resolution to your task (from high energy physics all way to astronomy). Panda's choice is only good for high-frequency stock traders, though.
- Bostonian 5y ago
  Pandas was started by a quant working for AQR Capital, so it's not surprising if "Panda's choice is only good for high-frequency stock traders".
  - lr1970 5y ago
    An illustrative example of how reasonable short-term and narrow-scope considerations can be really bad in long-term and/or at a larger scope.
    - nojito 5y ago
      Why should he care about other use-cases? It’s not his responsibility to make sure his package is as wide as possible before opensourcing.
      - lr1970 5y ago
        The problem is not with the Wes' original decision but with the fact that it was never revisited even when pandas took off at much larger scope. Should had been fixed before 1.0 release.
        nojito 5y ago
laichzeit0 5y ago
Great that they finally added quantile regression. This was sorely missed. I’m still hoping for a mixed-effects model implementation someday, like lme4 in R. The statsmodels implementation can only do predictions on fixed effects, which limits it greatly. I’ve always wondered why mixed effect type models are not more popular in the ML world.
- crimsoneer 5y ago
  Preach. The statsmodels implement sucks.
XoS-490 5y ago
What about sktime? https://github.com/alan-turing-institute/sktime https://github.com/alan-turing-institute/sktime

260 points by m3at 5y ago

zibzab 5y ago

Is anyone using scikit for NN? Why/why not?

armcat 5y ago
NN as in "neural network", or NN as in "nearest neighbour" algorithm? No to the former, yes to the latter. The reason for a "no" to neural networks - in my case I've only ever implemented neural networks with many layers, and typically using kernels, pooling mechanisms, etc, and since scikit-learn doesn't have GPU support, I opt for frameworks that do (PyTorch, TensorFlow). However, if you're only building fully-connected neural nets (MLPs), with just a few layers, you don't need GPU support since any benefits of having parallel processing are offset by shuffling data between CPU and GPU. So in that case, scikit-learn would probably work quite well, although I never tested this myself.
- woko 5y ago
  GPU can be useful for Nearest Neighbour as well. In case you have access to a GPU, I would strongly recommend Facebook's FAISS [1,2]. For everything else, sklearn is amazing. [1] https://faiss.ai/ https://faiss.ai/ [2] https://github.com/facebookresearch/faiss https://github.com/facebookresearch/faiss
  - armcat 5y ago
    Faiss looks very nice, thanks for the tip!
- zibzab 5y ago
  But ignoring acceleration, could one use scikit instead of tesnorflow? I mean, ignoring everything else, scikit has a much friendlier API.
jptech 5y ago
You should be asking if anyone is using it for ML, not NN.
- detaro 5y ago
  why?
FlyingSaucer 5y ago
I have used the MLP classifier[1] before. It's very simple to use (like most of sklearn's models). Worked well for standard and reasonably small classification model, but lacks some features for it to be a flexible way of using NNs: - No saving checkpoints (can be crucial for large models who need alot of compute and time) - No way to assign different activation functions to different layers - No complex nodes like LSTM, GRU - No way to implement complex architectures like transformers, encoders etc I also do not know if its even possible to use CUDA or any GPU with it. [1] : https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html#sklearn.neural_network.MLPClassifier https://scikit-learn.org/stable/modules/generated/sklearn.ne...
- kuu 5y ago
  I would say the same as you. As long as you only need a simple model, yes, the MLP is good enough, but forget about making any DeepLearning stuff. And AFAIK, there isn't GPU support, CPU performance is poor compared to GPU execution.
westurner 5y ago
There are scikit-learn (sklearn) API-compatible wrappers for e.g. PyTorch and TensorFlow. Skorch: https://github.com/skorch-dev/skorch https://github.com/skorch-dev/skorch tf.keras.wrappers.scikit_learn: https://www.tensorflow.org/api_docs/python/tf/keras/wrappers/scikit_learn https://www.tensorflow.org/api_docs/python/tf/keras/wrappers... AFAIU, there are not Yellowbrick visualizers for PyTorch or TensorFlow; though PyTorch abd TensorFlow work with TensorBoard for visualizing CFG execution. > Many machine learning libraries implement the scikit-learn `estimator API` to easily integrate alternative optimization or decision methods into a data science workflow. Because of this, it seems like it should be simple to drop in a non-scikit-learn estimator into a Yellowbrick visualizer, and in principle, it is. However, the reality is a bit more complicated. > Yellowbrick visualizers often utilize more than just the method interface of estimators (e.g. `fit()` and `predict()`), relying on the learned attributes (object properties with a single underscore suffix, e.g. `coef_`). The issue is that when a third-party estimator does not expose these attributes, truly gnarly exceptions and tracebacks occur. Yellowbrick is meant to aid machine learning diagnostics reasoning, therefore instead of just allowing drop-in functionality that may cause confusion, we’ve created a wrapper functionality that is a bit kinder with it’s messaging. Looks like there are Yellowbrick wrappers for XGBoost, CatBoost, CuML, and Spark MLib; but not for NNs yet. https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html#tested-libraries https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html... From the RAPIDS.ai CuML team: https://docs.rapids.ai/api/cuml/stable/ https://docs.rapids.ai/api/cuml/stable/ : > cuML is a suite of fast, GPU-accelerated machine learning algorithms designed for data science and analytical tasks. Our API mirrors Sklearn’s, and we provide practitioners with the easy fit-predict-transform paradigm without ever having to program on a GPU. > As data gets larger, algorithms running on a CPU becomes slow and cumbersome. RAPIDS provides users a streamlined approach where data is intially loaded in the GPU, and compute tasks can be performed on it directly. CuML is not an NN library; but there are likely performance optimizations from CuDF and CuML that would accelerate performance of NNs as well. Dask ML works with models with sklearn interfaces, XGBoost, LightGBM, PyTorch, and TensorFlow: https://ml.dask.org/ https://ml.dask.org/ : > Scikit-Learn API > In all cases Dask-ML endeavors to provide a single unified interface around the familiar NumPy, Pandas, and Scikit-Learn APIs. Users familiar with Scikit-Learn should feel at home with Dask-ML. dask-labextension for JupyterLab helps to visualize Dask ML CFGs which call predictors and classifiers with sklearn interfaces: https://github.com/dask/dask-labextension https://github.com/dask/dask-labextension
woko 5y ago
See the FAQ: https://scikit-learn.org/dev/faq.html#will-you-add-gpu-support https://scikit-learn.org/dev/faq.html#will-you-add-gpu-suppo...
- deleted 5y ago
  [deleted]

NeutralForest 5y ago

Excellent library with stellar documentation, I hope it'll live on for a long time.

jgilias 5y ago
Yes, glad they've decided it's finally out of the "don't use it, it's experimental" phase and gotten off the 0ver.org wall of shame!
Uberphallus 5y ago
Really, for any other ML library the best documentation is how-tos spread through the web, but scikit-learn leaves very little room for that kind of content.
sveme 5y ago
Best documented library. It even provides examples, guidance and best practices in the documentation. Have rarely learned so much as when I went through the sci-kit documentation. Absolute delight.
- mistrial9 5y ago
  you mean the 4000 page cookbook thing?

monkeybutton 5y ago

Scikit-Learn is great, and, reading the documentation for other 3rd party ML packages and seeing the words "Scikit-learn API" is even better.

lysecret 5y ago

Excellent library for train_test_split. Jokes aside. This next to Numpy, Pandas Jupyter and Matplotlib + the DL libraries are the reason Python is the powerhouse it is for Data Science.

disgruntledphd2 5y ago
I'm with you on sklearn, the DL libraries and Numpy, but Pandas and Matplotlib are poor, poor relations of the tools available in the R ecosystem (dplyr/ggplot etc).
- pc86 5y ago
  If you're doing data science aren't sklearn, DL, and numpy getting you 90% of the way there anyway? Even if R has better "versions" of pandas/matplotlib (not conceding that point) it's not exactly central to the job of data science.
  - civilized 5y ago
    As a working data scientist I'd say it's completely the opposite: a good tabular data manipulation package is the single most valuable tool in my tool box. And R's packages (either data.table or dplyr) are definitely way better than pandas. There's no comparison. I would be hard-pressed to find a working data scientist whose definition of data science is "that thing you do with sklearn, Deep Learning and Numpy".
    - jstx1 5y ago
      > "Data science is that thing where you do sklearn, Deep Learning and Numpy" is not a working data scientist's perspective. It could be. It's such a broad job title and it looks so different across different companies and teams that the main tool for one data scientist might be something that another data scientist never has to touch. Different data science jobs prioritise different tools, that's all.
      - 5y ago

infimum 5y ago

scikit-learn (next to numpy) is the one library I use in every single project at work. Every time I consider switching away from python I am faced with the fact that I'd lose access to this workhorse of a library. Of course it's not all sunshine and rainbows - I had my fair share of rummaging through its internals - but its API design is a de-facto standard for a reason. My only recurring gripe is that the serialization story (basically just pickling everything) is not optimal.

CapmCrackaWaka 5y ago
I recently ran into this issue as well. Serialization of sklearn random forests results in absolutely massive files. I had to switch to lightgbm, which is 100x faster to load from a save file and about 20x smaller.
zeec123 5y ago
There is so much wrong with the api design of sklearn (how can one think "predict_proba" is a good function name?). I can understand this, since most of it was probably written by PhD students without the time and expertise to come up with a proper api; many of them without a CS background.[1] [1] https://www.reddit.com/r/haskell/comments/7brsuu/machine_learning_in_functional_programming/dppck1x?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/haskell/comments/7brsuu/machine_lea...
- mrtranscendence 5y ago
  I didn't want to bag on sklearn (I've already bagged on pandas enough here), but for what it's worth I agree with you. It's, ahh, not the API I would've come up with. It's what everybody has standardized on, though, and maybe there's some value in that.
- kzrdude 5y ago
  These seem like minor gripes (reading your link) - and I don't even agree with them, seems like an ok use of mutable state (otherwise a separate object would be needed for hyperparameter state?). Maybe my expectations are low, but they way sklearn unifies the API across different estimators all across the library - that's already way above what you can expect - especially if you consider it to be "written by a bunch of phd students".

lr1970 5y ago

Just to clarify, scikit-learn 1.0 has not been released yet. The latest tag in the github repo is 1.0.rc2 https://github.com/scikit-learn/scikit-learn/releases/tag/1.0.rc2 https://github.com/scikit-learn/scikit-learn/releases/tag/1....

conor_f 5y ago

https://0ver.org/ https://0ver.org/ will need an update!

lr1970 5y ago

Early on, pandas made some unfortunate design decisions that are still biting hard. For example, the choice of datetime (pandas.Timestamp) represented by a 64-bit int with a fixed nanosecond resolution. This choice gives dynamic range of +- 292 years around 1970-01-01 (the epoch). This range is too small to represent the works of William Shakespeare, never mind human history. Using pandas in these areas becomes a royal pain in the neck, for one constantly needs to work around pandas datetime limitations. OTOH, in numpy one can choose time resolution units (anything from attosecond to a year) tailoring time resolution to your task (from high energy physics all way to astronomy). Panda's choice is only good for high-frequency stock traders, though.

Bostonian 5y ago
Pandas was started by a quant working for AQR Capital, so it's not surprising if "Panda's choice is only good for high-frequency stock traders".
- lr1970 5y ago
  An illustrative example of how reasonable short-term and narrow-scope considerations can be really bad in long-term and/or at a larger scope.
  - nojito 5y ago
    Why should he care about other use-cases? It’s not his responsibility to make sure his package is as wide as possible before opensourcing.
    - lr1970 5y ago
      The problem is not with the Wes' original decision but with the fact that it was never revisited even when pandas took off at much larger scope. Should had been fixed before 1.0 release.
      - nojito 5y ago

laichzeit0 5y ago

Great that they finally added quantile regression. This was sorely missed. I’m still hoping for a mixed-effects model implementation someday, like lme4 in R. The statsmodels implementation can only do predictions on fixed effects, which limits it greatly. I’ve always wondered why mixed effect type models are not more popular in the ML world.

crimsoneer 5y ago
Preach. The statsmodels implement sucks.

XoS-490 5y ago

What about sktime? https://github.com/alan-turing-institute/sktime https://github.com/alan-turing-institute/sktime