2 ms·
Anyone has more insight into why? if I try to grid search hyperparameters for a random forest takes ages in a single machine with scikit-learn. I only ever see
by lz400 9y ago
Anyone has more insight into why? if I try to grid search hyperparameters for a random forest takes ages in a single machine with scikit-learn. I only ever see GPUs in neural networks. Is there some acceleration for non-NN machine learning algos?
- bmh100 9y agoAs an aside, you should generally avoid grid search in favor of random search or an evolutionary algorithm. Grid search can waste your time budget on likely bad combinations of hyperparameters instead of looking for combinations local to or nearby a known-good combination.
- T-A 9y agoGenerally speaking, what GPUs excel at is applying exactly the same operation to many parallel data streams. Neural networks are like that. Random forests are the opposite: branching on conditional tests is very expensive. It's been several years since I last wrote raw CUDA and OpenCL, but if memory serves, back then the docs essentially said that every if...else amounted to running both branches on all the data and then deciding what to keep, effectively halving performance. So a decision tree just a few levels deep would slow you down by an order of magnitude.