7 ms·

Is anyone using scikit for NN? Why/why not?

by zibzab 5y ago

armcat 5y ago
NN as in "neural network", or NN as in "nearest neighbour" algorithm? No to the former, yes to the latter. The reason for a "no" to neural networks - in my case I've only ever implemented neural networks with many layers, and typically using kernels, pooling mechanisms, etc, and since scikit-learn doesn't have GPU support, I opt for frameworks that do (PyTorch, TensorFlow). However, if you're only building fully-connected neural nets (MLPs), with just a few layers, you don't need GPU support since any benefits of having parallel processing are offset by shuffling data between CPU and GPU. So in that case, scikit-learn would probably work quite well, although I never tested this myself.
- woko 5y ago
  GPU can be useful for Nearest Neighbour as well. In case you have access to a GPU, I would strongly recommend Facebook's FAISS [1,2]. For everything else, sklearn is amazing. [1] https://faiss.ai/ https://faiss.ai/ [2] https://github.com/facebookresearch/faiss https://github.com/facebookresearch/faiss
  - armcat 5y ago
    Faiss looks very nice, thanks for the tip!
- zibzab 5y ago
  But ignoring acceleration, could one use scikit instead of tesnorflow? I mean, ignoring everything else, scikit has a much friendlier API.
jptech 5y ago
You should be asking if anyone is using it for ML, not NN.
- detaro 5y ago
  why?
FlyingSaucer 5y ago
I have used the MLP classifier[1] before. It's very simple to use (like most of sklearn's models). Worked well for standard and reasonably small classification model, but lacks some features for it to be a flexible way of using NNs: - No saving checkpoints (can be crucial for large models who need alot of compute and time) - No way to assign different activation functions to different layers - No complex nodes like LSTM, GRU - No way to implement complex architectures like transformers, encoders etc I also do not know if its even possible to use CUDA or any GPU with it. [1] : https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html#sklearn.neural_network.MLPClassifier https://scikit-learn.org/stable/modules/generated/sklearn.ne...
- kuu 5y ago
  I would say the same as you. As long as you only need a simple model, yes, the MLP is good enough, but forget about making any DeepLearning stuff. And AFAIK, there isn't GPU support, CPU performance is poor compared to GPU execution.
westurner 5y ago
There are scikit-learn (sklearn) API-compatible wrappers for e.g. PyTorch and TensorFlow. Skorch: https://github.com/skorch-dev/skorch https://github.com/skorch-dev/skorch tf.keras.wrappers.scikit_learn: https://www.tensorflow.org/api_docs/python/tf/keras/wrappers/scikit_learn https://www.tensorflow.org/api_docs/python/tf/keras/wrappers... AFAIU, there are not Yellowbrick visualizers for PyTorch or TensorFlow; though PyTorch abd TensorFlow work with TensorBoard for visualizing CFG execution. > Many machine learning libraries implement the scikit-learn `estimator API` to easily integrate alternative optimization or decision methods into a data science workflow. Because of this, it seems like it should be simple to drop in a non-scikit-learn estimator into a Yellowbrick visualizer, and in principle, it is. However, the reality is a bit more complicated. > Yellowbrick visualizers often utilize more than just the method interface of estimators (e.g. `fit()` and `predict()`), relying on the learned attributes (object properties with a single underscore suffix, e.g. `coef_`). The issue is that when a third-party estimator does not expose these attributes, truly gnarly exceptions and tracebacks occur. Yellowbrick is meant to aid machine learning diagnostics reasoning, therefore instead of just allowing drop-in functionality that may cause confusion, we’ve created a wrapper functionality that is a bit kinder with it’s messaging. Looks like there are Yellowbrick wrappers for XGBoost, CatBoost, CuML, and Spark MLib; but not for NNs yet. https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html#tested-libraries https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html... From the RAPIDS.ai CuML team: https://docs.rapids.ai/api/cuml/stable/ https://docs.rapids.ai/api/cuml/stable/ : > cuML is a suite of fast, GPU-accelerated machine learning algorithms designed for data science and analytical tasks. Our API mirrors Sklearn’s, and we provide practitioners with the easy fit-predict-transform paradigm without ever having to program on a GPU. > As data gets larger, algorithms running on a CPU becomes slow and cumbersome. RAPIDS provides users a streamlined approach where data is intially loaded in the GPU, and compute tasks can be performed on it directly. CuML is not an NN library; but there are likely performance optimizations from CuDF and CuML that would accelerate performance of NNs as well. Dask ML works with models with sklearn interfaces, XGBoost, LightGBM, PyTorch, and TensorFlow: https://ml.dask.org/ https://ml.dask.org/ : > Scikit-Learn API > In all cases Dask-ML endeavors to provide a single unified interface around the familiar NumPy, Pandas, and Scikit-Learn APIs. Users familiar with Scikit-Learn should feel at home with Dask-ML. dask-labextension for JupyterLab helps to visualize Dask ML CFGs which call predictors and classifiers with sklearn interfaces: https://github.com/dask/dask-labextension https://github.com/dask/dask-labextension
woko 5y ago
See the FAQ: https://scikit-learn.org/dev/faq.html#will-you-add-gpu-support https://scikit-learn.org/dev/faq.html#will-you-add-gpu-suppo...
- deleted 5y ago
  [deleted]

Is anyone using scikit for NN? Why/why not?

by zibzab 5y ago

Is anyone using scikit for NN? Why/why not?

NN as in "neural network", or NN as in "nearest neighbour" algorithm? No to the former, yes to the latter. The reason for a "no" to neural networks - in my case I've only ever implemented neural networks with many layers, and typically using kernels, pooling mechanisms, etc, and since scikit-learn doesn't have GPU support, I opt for frameworks that do (PyTorch, TensorFlow). However, if you're only building fully-connected neural nets (MLPs), with just a few layers, you don't need GPU support since any benefits of having parallel processing are offset by shuffling data between CPU and GPU. So in that case, scikit-learn would probably work quite well, although I never tested this myself.

woko 5y ago
GPU can be useful for Nearest Neighbour as well. In case you have access to a GPU, I would strongly recommend Facebook's FAISS [1,2]. For everything else, sklearn is amazing. [1] https://faiss.ai/ https://faiss.ai/ [2] https://github.com/facebookresearch/faiss https://github.com/facebookresearch/faiss
- armcat 5y ago
  Faiss looks very nice, thanks for the tip!
zibzab 5y ago
But ignoring acceleration, could one use scikit instead of tesnorflow? I mean, ignoring everything else, scikit has a much friendlier API.

jptech 5y ago

You should be asking if anyone is using it for ML, not NN.

detaro 5y ago
why?

FlyingSaucer 5y ago

I have used the MLP classifier[1] before. It's very simple to use (like most of sklearn's models). Worked well for standard and reasonably small classification model, but lacks some features for it to be a flexible way of using NNs: - No saving checkpoints (can be crucial for large models who need alot of compute and time) - No way to assign different activation functions to different layers - No complex nodes like LSTM, GRU - No way to implement complex architectures like transformers, encoders etc I also do not know if its even possible to use CUDA or any GPU with it. [1] : https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html#sklearn.neural_network.MLPClassifier https://scikit-learn.org/stable/modules/generated/sklearn.ne...

kuu 5y ago
I would say the same as you. As long as you only need a simple model, yes, the MLP is good enough, but forget about making any DeepLearning stuff. And AFAIK, there isn't GPU support, CPU performance is poor compared to GPU execution.

westurner 5y ago

There are scikit-learn (sklearn) API-compatible wrappers for e.g. PyTorch and TensorFlow. Skorch: https://github.com/skorch-dev/skorch https://github.com/skorch-dev/skorch tf.keras.wrappers.scikit_learn: https://www.tensorflow.org/api_docs/python/tf/keras/wrappers/scikit_learn https://www.tensorflow.org/api_docs/python/tf/keras/wrappers... AFAIU, there are not Yellowbrick visualizers for PyTorch or TensorFlow; though PyTorch abd TensorFlow work with TensorBoard for visualizing CFG execution. > Many machine learning libraries implement the scikit-learn `estimator API` to easily integrate alternative optimization or decision methods into a data science workflow. Because of this, it seems like it should be simple to drop in a non-scikit-learn estimator into a Yellowbrick visualizer, and in principle, it is. However, the reality is a bit more complicated. > Yellowbrick visualizers often utilize more than just the method interface of estimators (e.g. `fit()` and `predict()`), relying on the learned attributes (object properties with a single underscore suffix, e.g. `coef_`). The issue is that when a third-party estimator does not expose these attributes, truly gnarly exceptions and tracebacks occur. Yellowbrick is meant to aid machine learning diagnostics reasoning, therefore instead of just allowing drop-in functionality that may cause confusion, we’ve created a wrapper functionality that is a bit kinder with it’s messaging. Looks like there are Yellowbrick wrappers for XGBoost, CatBoost, CuML, and Spark MLib; but not for NNs yet. https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html#tested-libraries https://www.scikit-yb.org/en/latest/api/contrib/wrapper.html... From the RAPIDS.ai CuML team: https://docs.rapids.ai/api/cuml/stable/ https://docs.rapids.ai/api/cuml/stable/ : > cuML is a suite of fast, GPU-accelerated machine learning algorithms designed for data science and analytical tasks. Our API mirrors Sklearn’s, and we provide practitioners with the easy fit-predict-transform paradigm without ever having to program on a GPU. > As data gets larger, algorithms running on a CPU becomes slow and cumbersome. RAPIDS provides users a streamlined approach where data is intially loaded in the GPU, and compute tasks can be performed on it directly. CuML is not an NN library; but there are likely performance optimizations from CuDF and CuML that would accelerate performance of NNs as well. Dask ML works with models with sklearn interfaces, XGBoost, LightGBM, PyTorch, and TensorFlow: https://ml.dask.org/ https://ml.dask.org/ : > Scikit-Learn API > In all cases Dask-ML endeavors to provide a single unified interface around the familiar NumPy, Pandas, and Scikit-Learn APIs. Users familiar with Scikit-Learn should feel at home with Dask-ML. dask-labextension for JupyterLab helps to visualize Dask ML CFGs which call predictors and classifiers with sklearn interfaces: https://github.com/dask/dask-labextension https://github.com/dask/dask-labextension

woko 5y ago

See the FAQ: https://scikit-learn.org/dev/faq.html#will-you-add-gpu-support https://scikit-learn.org/dev/faq.html#will-you-add-gpu-suppo...

deleted 5y ago
[deleted]