4 ms·

This is a fascinating concept: choosing the optimum training set for a "learner" when the true model is known. While the paper focuses on using the system to te

by bmh100 12y ago

This is a fascinating concept: choosing the optimum training set for a "learner" when the true model is known. While the paper focuses on using the system to teach humans, I see the value in other cases where the training set would be expensive to acquire. Examples would include long-running simulations, such as protein folding, or where each example has a significant materials cost, like chemical testing. Clearly, generating 60,000 observations (MNIST) would quickly become cost prohibitive compared to carefully selecting training examples to optimize learning.

letitgo12345 12y ago
Hmm for those cases though, you would have to parametrize the training set to be able to optimally select the next training example in which case you're back to active learning.
- bmh100 12y ago
  > The key difference is that the teacher knows theta-star upfront and doesn’t need to explore. Based on this quote from page 1 and Wikipedia for "active learning" [1], I see the distinction in my examples as this: 1. In machine teaching, the teacher chooses the new observations to be presented. In active learning, the learner chooses new observations to be labeled. 2. In machine teaching, the availability of observations in general is controlled. In active learning, observations may be available but unlabeled, as in a cross-validation set. E.g., voting on which observation to have labeled. What is your understanding on these topics? [1]: https://en.wikipedia.org/wiki/Active_learning_%28machine_learning%29 https://en.wikipedia.org/wiki/Active_learning_%28machine_lea...
Houshalter 12y ago
In these situations "active learning" is used. You generate multiple models, ideally with Bayesian inference, and then do a search for an example that causes the most disagreement among them.
- bmh100 12y ago
  Could you recommend an introductory resource on that particular process?

by bmh100 12y ago

letitgo12345 12y ago

Hmm for those cases though, you would have to parametrize the training set to be able to optimally select the next training example in which case you're back to active learning.

bmh100 12y ago
> The key difference is that the teacher knows theta-star upfront and doesn’t need to explore. Based on this quote from page 1 and Wikipedia for "active learning" [1], I see the distinction in my examples as this: 1. In machine teaching, the teacher chooses the new observations to be presented. In active learning, the learner chooses new observations to be labeled. 2. In machine teaching, the availability of observations in general is controlled. In active learning, observations may be available but unlabeled, as in a cross-validation set. E.g., voting on which observation to have labeled. What is your understanding on these topics? [1]: https://en.wikipedia.org/wiki/Active_learning_%28machine_learning%29 https://en.wikipedia.org/wiki/Active_learning_%28machine_lea...

Houshalter 12y ago

In these situations "active learning" is used. You generate multiple models, ideally with Bayesian inference, and then do a search for an example that causes the most disagreement among them.

bmh100 12y ago
Could you recommend an introductory resource on that particular process?