4 ms·
> simply requiring a way to go back and spend more effort later when required Present neural nets have no "more effort" or "less effort" knob. For the same inp
by 21 8y ago
> simply requiring a way to go back and spend more effort later when required
Present neural nets have no "more effort" or "less effort" knob. For the same input they always produce the same output.
The article says a different thing: humans rerun the "algorithm" again, but this time with the knowledge that something was wrong in the previous run (lets say color), and this knowledge will tweak the way it runs again, maybe even running a completely different algorithm the second time (a network more specialized in color but which is worse at shapes)
- taeric 8y agoNaively, doing an ensemble of different networks is effectively this. I don't know if folks doing that, though. Pretty sure these edge cases don't matter nearly as much as lack of data for most of us. Consider, can you recognize that distant relative you have never seen? Why not?
- p1esk 8y agoYou can easily add a "knob" like this to any model. For example, you can first run an input through a fast, but less accurate object detector (e.g. YOLO), and if not satisfied with results (or automatically triggered by low confidence scores), run the same input through a slower, but more accurate model (RCNN).
- p1esk 8y agoYou can also have a "generalist" model, trained to differentiate major object classes (e.g. furniture from animals), and multiple "specialist" models, each trained to differentiate between variants of the same object category (e.g. breeds of dogs). That approach was described in Hinton's knowledge distillation paper.