3 ms·
I am trying to speed up a machine learning algorithm. There are a very large number of possible output classes. It would be too expensive on each training step
by bravura 18y ago
I am trying to speed up a machine learning algorithm.
There are a very large number of possible output classes.
It would be too expensive on each training step to perform an update for all outputs.
I use an exponential moving average to keep track of an estimate of which outputs are correctly learned (low weight) and which are not well-learned (high weight). In this way, I can focus training on the what it currently has not learned well, and not spend training time on what it already knows.
- loicfevrier 18y agoWhich language are you using ? Matlab, C/C++ ? What would be an estimate of N : 10^3, 10^6, 10^9 ?
- bravura 18y agopython/numpy with C. Funny story, that. At our lab, we are developing an optimizing compiler in Python for math expressions. You write the function you want to repeatedly evaluate in Python. Maybe there's a gradient thrown in. Our library optimizes the function graph, converts in to C, and compiles it. You can then execute your function in python, but it's fast. We should be releasing 0.1 in a week.