2 ms·

> Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Any resources on that? I have a feel

> Now we have a fairly concrete idea of what a potential AGI might look like - an RL agent that uses a large transformer. Any resources on that? I have a feeling that RL might play a big role in the first AGI, too, but why transformers in particular?

moultano 4y ago
Transformers have gradually taken over in every other ML domain.
- lagrange77 4y ago
  Okay, but do those ML domains help with AGI?
gmadsen 4y ago
they don't seem to have a theoretical upper limit. more data and more parameters seem to just keep making it more advanced. Even in ways that weren't predicted or understood. the difference between a language model that can explain a novel joke and one that can't is purely scale. So the thought is with enough scale, you eventually hit AGI
Isinlor 4y ago
See: https://arxiv.org/abs/2205.06175 https://arxiv.org/abs/2205.06175 A Generalist Agent Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato. Gato is a 1 to 2 billion parameters model due to latency considerations in real time physical robots usage. So for today standards of 500 billion parameters dense models Gato is tiny. Additionally Gato is trained on data produced by other RL agents. It did not do the exploration fully itself. Demis Hassabis say that DeepMind is currently working on Gato v2.
Jack000 4y ago
Everything Deepmind published at this year's ICML would be a good start. Transformers (or rather the QKV attention mechanism) has taken over ML research at this point, it just scales and works in places it really shouldn't. Eg. you'd think convnets would make more sense for vision because of its translation invariance, but ViT works better even without this inductive bias. Even in things like diffusion models the attention layers are crucial to making the model work.

moultano 4y ago

Transformers have gradually taken over in every other ML domain.

lagrange77 4y ago
Okay, but do those ML domains help with AGI?

gmadsen 4y ago

they don't seem to have a theoretical upper limit. more data and more parameters seem to just keep making it more advanced. Even in ways that weren't predicted or understood. the difference between a language model that can explain a novel joke and one that can't is purely scale. So the thought is with enough scale, you eventually hit AGI

Isinlor 4y ago

See: https://arxiv.org/abs/2205.06175 https://arxiv.org/abs/2205.06175 A Generalist Agent Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato. Gato is a 1 to 2 billion parameters model due to latency considerations in real time physical robots usage. So for today standards of 500 billion parameters dense models Gato is tiny. Additionally Gato is trained on data produced by other RL agents. It did not do the exploration fully itself. Demis Hassabis say that DeepMind is currently working on Gato v2.

Jack000 4y ago

Everything Deepmind published at this year's ICML would be a good start. Transformers (or rather the QKV attention mechanism) has taken over ML research at this point, it just scales and works in places it really shouldn't. Eg. you'd think convnets would make more sense for vision because of its translation invariance, but ViT works better even without this inductive bias. Even in things like diffusion models the attention layers are crucial to making the model work.