4 ms·
A 5000x boost in KNN inference is not bad. Generally speaking the distribution-packaged versions of python and all its scientific libraries and their support l
by jeffbee 5y ago
A 5000x boost in KNN inference is not bad.
Generally speaking the distribution-packaged versions of python and all its scientific libraries and their support libraries are best ignored. That stuff should always be rebuilt to suit your actual production hardware, instead of a 2007-era Opteron.
- tomrod 5y agoIs there a way via pip/conda to compile these to your environment directly? I see most people just pull from repositories and sometimes see wheel discussed.
- peter_retief 5y agoconda install scikit-learn-intelex -c conda-forge
- tomrod 5y agoThanks Peter. I thought conda-forge was a repository/channel, not a command to compile to local environment. A few followups: (1) Is this usable for non-intelex packages? (2) What about packages not in conda's channels?
- peter_retief 5y agoI am busy installing it now. Anaconda should take care of required packages? I am not actually sure. Seems to be working. Intel(R) Extension for Scikit-learn* enabled (https://github.com/intel/scikit-learn-intelex https://github.com/intel/scikit-learn-intelex)
- deleted 5y ago[deleted]
- gojomo 5y agoIs there a demo that shows this 5000x speedup? (A jump that large suggests to me they may be fixing issues in the default implementation that could also be fixed for other processors!)
- blt 5y agoLooks like they are responding to https://github.com/intel/scikit-learn-intelex#-acceleration https://github.com/intel/scikit-learn-intelex#-acceleration I completely agree. I hope some Intel competitor funds a scikit-learn developer to read this code and extract all the portable performance improvements.
- ogrisel 5y agoAs @jjerphan commented above, there is already ongoing effort to get an improved portable brute-force implementation in vanilla scikit-learn, see: https://news.ycombinator.com/item?id=29069760 https://news.ycombinator.com/item?id=29069760
- napetrov 5y agoThe point is that sklernex would bring performance for all X86 architectures, not just Intel. And yes scikit developers already working on generic improvements there