3 ms·

but the DRL framework encompasses all the standard classification/regression tasks and also includes decision making Except labelling data, correct ? which is

but the DRL framework encompasses all the standard classification/regression tasks and also includes decision making Except labelling data, correct ? which is non trivial effort.

gwern 9y ago
> Except labelling data, correct ? which is non trivial effort. Choosing choice of labeling is a RL problem too. If you're choosing environment actions to learn how the environment 'labels' them, that's the classic topic of 'how do we make DRL models explore' well (and arguably is the Achilles heel of the model-free DRL approaches OP is criticizing: the NNs can easily learn to optimize their actions, even tiny NNs are more than enough, but they just don't get fed the 'right' data ie. exploration is bad). Relevant papers: https://www.reddit.com/r/reinforcementlearning/search?q=flair%3AExp&sort=new&restrict_sr=on https://www.reddit.com/r/reinforcementlearning/search?q=flai... If you're being very narrow and considering a classification problem, well, that's a RL problem too: you can optimize which datapoints you get labels for based on how informative a datapoint is (most datapoints are simply redundant) or how expensive it is to label. That's called 'active learning': https://www.reddit.com/r/reinforcementlearning/search?q=flair%3AActive&sort=new&restrict_sr=on https://www.reddit.com/r/reinforcementlearning/search?q=flai... It's particularly natural if you are doing large-scale image classification and have a service like Amazon Turk plugged in to get (or correct) labels.
PeterisP 9y ago
In many cases people greatly overestimate the effort of labelling data. For the price of just a single engineer man-month you can generally label a lot of data, sufficient for many tasks; especially if you can do bootstrapping (the humans don't label all data, but correct mistakes of a previous insufficiently good automated solution) or some transfer learning from a somewhat similar task or an unsupervised one. Like, if you need a solution for a some niche of image classification, even a single afternoon of labeling data might be sufficient to adapt an ImageNet classifier to your particular labels and get reasonable accuracy.

but the DRL framework encompasses all the standard classification/regression tasks and also includes decision making Except labelling data, correct ? which is non trivial effort.

gwern 9y ago

> Except labelling data, correct ? which is non trivial effort. Choosing choice of labeling is a RL problem too. If you're choosing environment actions to learn how the environment 'labels' them, that's the classic topic of 'how do we make DRL models explore' well (and arguably is the Achilles heel of the model-free DRL approaches OP is criticizing: the NNs can easily learn to optimize their actions, even tiny NNs are more than enough, but they just don't get fed the 'right' data ie. exploration is bad). Relevant papers: https://www.reddit.com/r/reinforcementlearning/search?q=flair%3AExp&sort=new&restrict_sr=on https://www.reddit.com/r/reinforcementlearning/search?q=flai... If you're being very narrow and considering a classification problem, well, that's a RL problem too: you can optimize which datapoints you get labels for based on how informative a datapoint is (most datapoints are simply redundant) or how expensive it is to label. That's called 'active learning': https://www.reddit.com/r/reinforcementlearning/search?q=flair%3AActive&sort=new&restrict_sr=on https://www.reddit.com/r/reinforcementlearning/search?q=flai... It's particularly natural if you are doing large-scale image classification and have a service like Amazon Turk plugged in to get (or correct) labels.

PeterisP 9y ago

In many cases people greatly overestimate the effort of labelling data. For the price of just a single engineer man-month you can generally label a lot of data, sufficient for many tasks; especially if you can do bootstrapping (the humans don't label all data, but correct mistakes of a previous insufficiently good automated solution) or some transfer learning from a somewhat similar task or an unsupervised one. Like, if you need a solution for a some niche of image classification, even a single afternoon of labeling data might be sufficient to adapt an ImageNet classifier to your particular labels and get reasonable accuracy.