ogrisel

26 ms·

211.
by ogrisel 13y ago
Prediction accuracy cannot be computed in a purely unsupervised setting: you don't have labels for the samples. You can compare the cluster found by k-means run several times against what you get with a randomized version of your dataset th
212.
by ogrisel 13y ago
This diagram is only presenting algorithms as currently implemented in scikit-learn. Deep Learning models are not yet implemented in scikit-learn although baselines (stacked RBMs and MLP) will probably make it in the coming months.
213.
by ogrisel 14y ago
Expectation Maximization is a very generic inference scheme that has many declination depending on the structure of the models it is applied to. For instance there we already have Gaussian Mixture Models that can be fitted using EM in sklea
214.
by ogrisel 14y ago
The scikit-learn developers are also interested in knowing more about organizations using the library for building innovative products or solving data related problems. If you are interested in writing a short paragraph about how your organ
215.
Scikit-learn 0.13: Nystroem method, hashing trick, faster random forests(peekaboo-vision.blogspot.de)
46 points by ogrisel 14y ago|3 comments
216.
by ogrisel 14y ago
Has anybody found / done a benchmark that would compare the scalability of a Cassandra cluster vs an ElasticSearch cluster the latter used as a NoSQL database (with stored fields)? I am interested with 2 kinds of scalability: - volume scala
217.
by ogrisel 14y ago
Other blogs by sklearners: http://peekaboo-vision.blogspot.fr/ http://www.mblondel.org/journal/category/machine-learning/ http://gael-varoquaux.info/blog/?cat=20 http://fseoane.net/blog/ http://blog.vene.ro/
218.
by ogrisel 14y ago
You should directly use Mahout: the recsys part is quite complete and high level and application oriented contrary to scikit-learn which does not provide high level recsys concepts. The best documentation I found is the Mahout in Action boo
219.
by ogrisel 14y ago
You could use scikit-learn to build your own recommender system if you really understand the math of the models you want to implement: scikit-learn will only provide you with low level building blocks such as a semi-scalable Singular Value
220.
by ogrisel 14y ago
Hadoop can use various fast compressors such as snappy, LZO, LZ4 so as to be able to read and store data from and to the hard drives faster than if uncompressed.
221.
by ogrisel 14y ago
Indeed for in-memory decompression LZ4 is much faster as expected: $ time ./LZ4Demo.exe -d /tmp/out.lz4 /dev/null *** Compression CLI using LZ4 algorithm , by Yann Collet (Jul 25 2012) *** Successfully decoded 3590403790 bytes Do
222.
by ogrisel 14y ago
Here are the decompression timings: $ time ./6unpack /tmp/out.fastlz Archive: /tmp/out.fastlz alpha.txt [..................................................] real 2m1.514s user 0m12.038s sys 0m16.199s $ time ./LZ
223.
by ogrisel 14y ago
7zip is a slow compressor that compresses well. fastlz should better be compared to LZ4 or Snappy which are meant to be trade compression rate for speed.
224.
by ogrisel 14y ago
It does compress slightly faster and better than lz4 (when the output is written to disk on a SSD) using the default parameters: For an input text file alpha.txt whose size is 3.3GB: - fastlz compresses in 1m6.446s (real time) $ time ./
225.
by ogrisel 14y ago
matlab and python + numpy + scipy + matplotlib give the same kind of performance for the same level of features and ability to do quick exploratory prototyping with high level, vector based datastructures. However one is free (beer and free
226.
by ogrisel 14y ago
Yes as long as you profile and make sure that the bottleneck is a blas / atlas call (when doing numpy vector operation) or a compiled extension of your own (e.g. a piece of Cython or c code) if you need a for loop that cannot be vectorized
227.
by ogrisel 14y ago
I think that in this case the data was generated in a uniform manner (centered and with isotropic variance). So it might not greatly impact the performance. ~50% training error is about the best you can get in a completely non-linearly sepa
228.
by ogrisel 14y ago
One unmentioned caveat of SGD is how to configure the learning rate schedule. scikit-learn is using Bottou's tricks that seem to work reasonably well in practice but it might even be better to implement the online estimate of optimal learni
229.
by ogrisel 14y ago
Have a look at the RCV1 benchmarks on this page: http://leon.bottou.org/projects/sgd SGD is still slightly faster but liblinear is behaving good enough in that case.
230.
by ogrisel 14y ago
The author of the blog post has promised more extensive tests with other datasets in future posts. Also there is an ongoing GSoC project for setting up a systematic benchmark infrastructure + performance optimizations for the scikit-learn p
231.
by ogrisel 14y ago
SGD with averaging [1] :) More seriously, SGD is pretty hard to beat for fitting linear models (SVM, logistic regression and other l1 penalized models with various loss functions) when the number of samples getting large. [1] http://leon.b
232.
by ogrisel 14y ago
As noted on the scikit-learn mailing list the poor results of the liblinear can be cause by a low convergence tolerance but also by the fact that the internal memory layout used by liblinear is not optimized for dense input data arrays as i
233.
Which machine learning classifiers are fast enough for medium-sized data?(blog.explainmydata.com)
50 points by ogrisel 14y ago|13 comments
234.
by ogrisel 14y ago
I did not know about MARS, do you have any practical benchmarking dataset / sample tasks where MARS is state of the art? What kind of regression problem do you use it for yourself? What is the typical dimension of the problems it can addres
235.
by ogrisel 15y ago
Thanks for the reply. I was told by twitter that this is similar to the kernel perceptron which I don't know well either. There is a good introduction with python code snippet here: http://www.mblondel.org/journal/2010/10/31/kernel-percept
236.
by ogrisel 15y ago
No actually it's not the case. But I don't know how Pegasos can be adapted to use kernels. If you take figure 1 of the paper [1] you will see that the gradient of the objective function is used to update a single weights vector `w` at each
237.
by ogrisel 15y ago
I agree that deep-learning is an interesting approach to learn higher level features. However it's still a long way from being a universal solution: for instance deep learning won't help you solve the machine translation or multi-documents
238.
by ogrisel 15y ago
Very nice article Aria. You quickly mention Pegasos as a scalable alternative to SMO. I agree that this works well for linear models. But despite the claim that Pegasos can be trivially adapted to kernel models I have never seen any impleme
239.
by ogrisel 15y ago
Incanter: http://incanter.org/
240.
by ogrisel 15y ago
Also, implementing ideas and testing them on real data can be a very good heuristic to drive theoretical thinking by: - giving you empirical evidences on the practical limitations of a positive theoretical results that might have been other

More ›