3 ms·
So transformers have done it again, another sub-field of ML with all its past approaches surpassed by a simple language model, at least when there is enough dat
by visarga 5y ago
So transformers have done it again, another sub-field of ML with all its past approaches surpassed by a simple language model, at least when there is enough data.
So far they can handle: text, image, video, code, proteins and now planning and behavior. It's like a universal algorithm for learning and reminds me of the uniformity of the brain. Hope we're going to see much more efficient hardware implementations in the future.
- blovescoffee 5y agoI wouldn't say they've "done it" quite yet. There's definitely an application for imitation learning but that might be it. A translation of the work in sequence-to-sequence to sequence-to-action is something I've also considered researching. A few challenges exist which the author touches on in just one sentence. First, we need data about previous sequences of actions and this is necessarily a challenge in many fields in robotics/learning. A related problem is that of exploration. How exactly should we inform the exploration of new sequences? Also, if our policy is based on the prediction of a Transformer, does it have the traditional desirable properties of a policy in an RL environment? Off the top of my head it seems like a Transformer fed into an MLP would probably be fit but I'm not sure. Transformers do seem promising, but it's a bit early to say they've "done it" :)
- eli_gottlieb 5y agoIt only really works if your domain has a neat discretization into discontinuous symbol sequences.
- deleted 5y ago[deleted]