9 ms·

>However, just because an AI doesn't understand a given input form, does not mean that it could not given adequate exposure; but it will need time to adapt and

>However, just because an AI doesn't understand a given input form, does not mean that it could not given adequate exposure; but it will need time to adapt and the closer the new information form is to one that it already understands, the faster it will incorporate the new understandings. That is just completely wrong. There is absolutely no reason to believe that some language model will produce the same (or even similar) accurracy and training speed, if trained with image data as additional input. What you are claiming here is that any (sufficiently complex) neural net can be continually fed with different problem data and will output sufficiently accurate results. There is no reason why this should be the case and for the millions of nets which have been created none of them have exhibited this property. It might very easily be the case that training GPT-3 with images means millions of times slower learning speeds for the same combined accuracy.

yarg 4y ago
> That is just completely wrong. There is absolutely no reason to believe ... yada yada ... none of them have exhibited this property. Go read up on pre-trained networks.
rictic 4y ago
Are you familiar with the Gato paper from DeepMind this May? I say this not as a gotcha, but as a genuine question that I'm interested in your response to. > Inspired by progress in large-scale language modelling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. > During the training phase of Gato, data from different tasks and modalities are serialised into a flat sequence of tokens, batched, and processed by a transformer neural network similar to a large language model. The loss is masked so that Gato only predicts action and text targets. https://www.deepmind.com/blog/a-generalist-agent https://www.deepmind.com/blog/a-generalist-agent

yarg 4y ago

> That is just completely wrong. There is absolutely no reason to believe ... yada yada ... none of them have exhibited this property. Go read up on pre-trained networks.

rictic 4y ago

Are you familiar with the Gato paper from DeepMind this May? I say this not as a gotcha, but as a genuine question that I'm interested in your response to. > Inspired by progress in large-scale language modelling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. > During the training phase of Gato, data from different tasks and modalities are serialised into a flat sequence of tokens, batched, and processed by a transformer neural network similar to a large language model. The loss is masked so that Gato only predicts action and text targets. https://www.deepmind.com/blog/a-generalist-agent https://www.deepmind.com/blog/a-generalist-agent