16 ms·
Intel Extension for Scikit-Learn
- jeffbee 5y agoA 5000x boost in KNN inference is not bad. Generally speaking the distribution-packaged versions of python and all its scientific libraries and their support libraries are best ignored. That stuff should always be rebuilt to suit your actual production hardware, instead of a 2007-era Opteron.
- tomrod 5y agoIs there a way via pip/conda to compile these to your environment directly? I see most people just pull from repositories and sometimes see wheel discussed.
- peter_retief 5y agoconda install scikit-learn-intelex -c conda-forge
- tomrod 5y agoThanks Peter. I thought conda-forge was a repository/channel, not a command to compile to local environment. A few followups: (1) Is this usable for non-intelex packages? (2) What about packages not in conda's channels?
- peter_retief 5y agoI am busy installing it now. Anaconda should take care of required packages? I am not actually sure. Seems to be working. Intel(R) Extension for Scikit-learn* enabled (https://github.com/intel/scikit-learn-intelex https://github.com/intel/scikit-learn-intelex)
- deleted 5y ago[deleted]
- gojomo 5y agoIs there a demo that shows this 5000x speedup? (A jump that large suggests to me they may be fixing issues in the default implementation that could also be fixed for other processors!)
- blt 5y agoLooks like they are responding to https://github.com/intel/scikit-learn-intelex#-acceleration https://github.com/intel/scikit-learn-intelex#-acceleration I completely agree. I hope some Intel competitor funds a scikit-learn developer to read this code and extract all the portable performance improvements.
- ogrisel 5y agoAs @jjerphan commented above, there is already ongoing effort to get an improved portable brute-force implementation in vanilla scikit-learn, see: https://news.ycombinator.com/item?id=29069760 https://news.ycombinator.com/item?id=29069760
- napetrov 5y agoThe point is that sklernex would bring performance for all X86 architectures, not just Intel. And yes scikit developers already working on generic improvements there
- kvathupo 5y agoIntel seems 6 years too late to the party CUDA started. That said, it could pick up traction: academics have increasingly been using pytorch. EDIT: Perhaps its my inexperience, but is anyone else confused by the OneAPI rollout? There isn't exactly backwards compatiblity with the Classic Intel compiler, and an embarassing amount of time elapsed until I realized "Data Parallel C++" doesn't refer to parallel programming in C++, but rather an Intel-developed API built atop C++.
- fartcannon 5y agoIf they're more open with it than nvidia, they have a chance in my opinion.
- dfsegoat 5y agoThey also made some entry into the R space with adding their MKL / BLAS library: https://www.intel.com/content/www/us/en/developer/articles/technical/using-onemkl-with-r.html https://www.intel.com/content/www/us/en/developer/articles/t...
- deleted 5y ago[deleted]
- mumblemumble 5y agoPerhaps things have changed since I last poked at this, so, standard disclaimers, take my comments with a grain of salt, etc. GPU acceleration is not a magic "go fast" machine. It only works for certain classes of embarrassingly parallel algorithms. In a nutshell, the parallel regions need to be long enough that the speedup from doing them in the GPU's silicon outweighs the relatively high cost of getting data into and out of the GPU. That's a fairly easy scenario to achieve with neural networks, which have a pretty high math-to-data ratio. Other machine learning algorithms, not necessarily. But basically all of them can benefit from the CPU's vector instructions, because they live in the CPU rather than out on a peripheral, so there's no hole you need to dig yourself out of before they can deliver a net benefit. I would also say that what academics are doing is not necessarily a good barometer for what others are doing. In another nutshell, academics' professional incentives encourage them to prefer the fanciest thing that could possibly work, because their job is to push the frontiers of knowledge and technology. Most people out in industry, though, are incentivized to do the simplest thing that could possibly work, because their job is to deliver software that is reliable and delivers a high return on investment.
- deleted 5y ago[deleted]
- joshlk 5y agoWhat changes have been made to get the speedups?
- napetrov 5y agovectorization, threading, cache optimizations, etc...
- savant_penguin 5y agoAs cool as this is, why would you lock yourself into Intel? Especially with cloud providers making arm processors available at lower prices. At the same time: "Intel® Extension for Scikit-learn* is a free software AI accelerator that brings over 10-100X acceleration across a variety of applications." Maybe their free software could be extended to all processors?
- bingohbangoh 5y agoI know people keep saying Intel is dead, but it's not entirely accurate imo. All of my machines still use Intels (other than my SBCs). So installing this and running it is trivial. Intel is still a major contributor to the Linux kernel. Thus, all their CPUs have first-class support for it. AMD fired all their Linux engineers some time back. They never rehired them to my knowledge. Then there's things like this (MKL libraries are another). Intel spends a lot more money on development of these little libraries which does meaningfully speed up processes. Those processes affect my day-to-day work as a software engineer. That adds up when I have to deploy on the cloud. ARM is not quite there yet and little hiccups at deploy time are a pain when the cost difference is not so significant relative to the hourly cost of my time. Linus Torvalds pointed this out about ARM, stating it couldn't ever take off unless it took off on the desktop.
- marcellus23 5y agoI don't think anyone is saying Intel is actually currently dead. They're clearly not. But their trajectory is not headed the right way.
- llampx 5y ago> AMD fired all their Linux engineers some time back. They never rehired them to my knowledge. My understanding is that AMD regularly contributes to the Linux kernel for their CPU and GPU lines. How would they do this without Linux engineers?
- formerly_proven 5y agoAMD has had multiple hiring rounds for Linux kernel engineers and their efforts regarding GPU support were never interrupted, so I dunno where you got that AMD fired "all their Linux engineers".
- evanb 5y agoThe syntax and usability of from sklearnex import patch_sklearn # The names match scikit-learn estimators patch_sklearn("SVC") seems quite clunky. I'd have preferred a syntax like from sklearnex import SVC Then, maintenance would be substantially easier. If sklearnex had import-level compatibility with sklearn it'd be as simple as some simple replacements, import sklearn --> import sklearnex as sklearn from sklearn.cluster import KMeans --> from sklearnex.cluster import KMeans which seems much easier / clearer.
- sdenton4 5y agoI really really loathe import magics. You end up with situations where dependencies change global behavior without a way to track down where the change is actually coming from.
- sillysaurusx 5y agoImport magic ends up causing all kinds of problems. There's no way to tell it "I just want to import your classes; I don't want you to patch!" without literally patching the library. In general, I'm a fan of "let me call the initializer myself, at program startup." It's especially important when you want reversibility, i.e. teardown in addition to initialization, which pops up all the time for unit tests.
- dangerbird2 5y agoAlso networking/web apps with lifecycle hooks, where careless import-time logic can break the setup procedures. To quote zen of python, "explicit is better than implicit"
- napetrov 5y agopatching is just one of options to enable optimizations - You also can do already sklearnex import SVC There are other ways how you can enable this https://intel.github.io/scikit-learn-intelex/what-is-patching.html https://intel.github.io/scikit-learn-intelex/what-is-patchin...
- streamofdigits 5y agoAccelerating scikit-learn is a smart move. At the algorithmic level for every ML use case there is probably x 10 non-ML data science projects. Also, it is good to have a true community framework that does not depend on the success of the metaverse for funding ;-) The lock-in is an important consideration, but if the scikit-learn API is fully respected it would seem less relevant. It also suggests a pattern for how other hardware vendors could accelerate scikit-learn as a genuine contribution?
- medo-bear 5y agoi personally think that it would be a more interesting move in its foray into hardware acceleration if intel gives first class support to julia
- AstroDogCatcher 5y agoIntel are focused on data-parallel C++ for delivering high performance, rightly or wrongly. Julia is one of those "nice in theory" options which has failed to live up to the hype and at this point seems unlikely to unseat python for most use-cases; it just doesn't have a good enough UX when used as a general purpose language.
- medo-bear 5y agoim not sure what you mean by UX in this context, but julias ecosystem for scientific computing (in a broad sense) has been growing tremendously. this is the area from which it wants to unseat python. general purpose programming is secondary. whether it can i don't know. but i definitely dont think its a settled question. python is my daily driver for machine learning work, but i definitely think julia can overtake its place eventually
- eduardosalaz 5y agoHi, I would love to hear more about your complaints regarding UX for general purpose programming
- klelatti 5y agoIs this Intel only or oneAPI which is supposed to be cross platform. Not entirely clear which makes me suspicious.
- jjerphan 5y agoHi all, Currently some work is being done to improve computational primitives of scikit-learn to enhance its overhaul performances natively. You can have a look at this exploratory PR: https://github.com/scikit-learn/scikit-learn/pull/20254 https://github.com/scikit-learn/scikit-learn/pull/20254 This other PR is a clear revamp of this previous one: https://github.com/scikit-learn/scikit-learn/pull/21462 https://github.com/scikit-learn/scikit-learn/pull/21462 Cheers, Julien.
- zibzab 5y agoAny idea if this can be used with Jupyter? I have a bunch of notebooks that take 4-8 hours to run. This could potentially make my life much easier.
- kzrdude 5y agoNo reason not to try it
- prionassembly 5y agoYou can always export jupyter notebooks to straight .py files. Also: how in th
- napetrov 5y agoyes, notebooks have no difference for sklernex You likely not exporting algorithms from scikit learn again after patching - i.e. patch call should be made prior to import. Note there are kaggle notebooks that showcase same optimizations - https://www.kaggle.com/napetrov/tps04-svm-with-intel-extension-for-scikit-learn https://www.kaggle.com/napetrov/tps04-svm-with-intel-extensi... They are also basically notebooks
- freediver 5y agoJust tried the patch in Google Colab and results for the example code were actually about 20% slower than without the patch. https://imgur.com/a/7EmlYJy https://imgur.com/a/7EmlYJy What am I missing? edit: it seems my instance was using AMD EPYC.
- napetrov 5y agoYou have to import KMeans again after patching. Also in you example it's tiny problem size that have lot of fluctuations. Basically in you code you are running stock both times. this extension also would bring perf to AMD, although Intel would be better optimized
- syntaxing 5y agoIs there a specific “test” to run as a performance standard for scikit? I noticed this the other day that my Mac mini M1 absolutely blows away my MacBook Air 2020 with an i7. I was always curious if there was a good way to gauge performance.
- klelatti 5y ago> oneAPI Data Analytics Library (oneDAL) is a powerful machine learning library that helps speed up big data analysis. oneDAL solvers are also used in Intel Distribution for Python for scikit-learn optimization. > oneDAL is part of oneAPI. So oneAPI is cross industry but this only works with Intel CPUs? Hmm. Not sure I’m buying this Intel. Sounds like you’re claiming to be open but locking people into Intel only libraries.
- westurner 5y agohttps://github.com/intel/scikit-learn-intelex https://github.com/intel/scikit-learn-intelex CuML is similar to Intel Extension for Scikit-Learn in function? https://github.com/rapidsai/cuml https://github.com/rapidsai/cuml > cuML is a suite of libraries that implement machine learning algorithms and mathematical primitives functions that share compatible APIs with other RAPIDS projects. cuML enables data scientists, researchers, and software engineers to run traditional tabular ML tasks on GPUs without going into the details of CUDA programming. In most cases, cuML's Python API matches the API from scikit-learn. For large datasets, these GPU-based implementations can complete 10-50x faster than their CPU equivalents. For details on performance, see the cuML Benchmarks Notebook.