24 ms·

DeepMind: A Generalist Agent

502 points by extr 4y ago

mrfusion 4y ago
I’m confused. Do the different modalities compliment each other? Can it learn more from text and images than text alone? Can you ask it to to draw a picture of a cat with the robot arm?
karmasimida 4y ago
Would this agent able to handle simple elementary mathematics? If they are using inspiration from Transformer, then it probably won't be able to count. For that, I don't really feel that enthusiastic about the 'Generalist' claim, maybe they think this is more catchy than just 'Multi-tasking'?
productceo 4y ago
Impressive
extr 4y ago
Abstract: Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato. Direct Link to Paper: https://dpmd.ai/Gato-paper https://dpmd.ai/Gato-paper
- blueberrychpstx 4y ago
  > we refer to as Gato First, humanity built enormous statues worshiping cats. Then, we let cats populate the largest amount of "image-bits" on the Internet. Now, we name the next closest thing to general AI after them. These damn felines sure are mysterious.
  - riwsky 4y ago
    it's all because cats made it so that, on the Internet, nobody knows you're a dog
- deleted 4y ago
  [deleted]
Ninjinka 4y ago
This seems huge, am I overestimating the significance?
- sdwr 4y ago
  Yeah.
- npwr 4y ago
  It is very impressive. Personnaly I'm still waiting for the unification of QM and GR. Also the adaptative nanobots that reconfigure our immune systems in real time.
- ravi-delia 4y ago
  This particular model is super bad at 600 deferent tasks. At it's size you'd expect it to be mediocre at best at even one of them, so it's still very impressive. Fascinating research, can't wait to see if it's generalizing and how, not sure how overall significant it is
- tomatowurst 4y ago
  Basically the achievement here is that they have produced a generic AI capable of engaging in different activities, and from here if we extrapolate, it could lead to even more refinement, wider range of activities with even more dimensions of complexity. It's reverting to replace somebody sitting in front of a screen, not just artists and coders but literally anything you can do on a screen which also means manipulation of remote hardware in the real world. Very possible that within our lifetime our networked OS would be able to perform much of these generalist tasks and content creation. I say OS because theres only a few companies that own the datacenters, software and hardware ecosystem to automate, and capital to invest big in a final mile innovation: Imagine playing Battlefield 15 with realistic and chatty AI while generating Sopranos Season 9 featuring Pauli Gaultieri Jr. with crowdsourced online storyboard to 8k film, while the same AI could be used to scalp money on Google Playstore by generating ad filled free versions of existing productivity apps that it reverse engineered, while your robot maids takes out the trash, cook you a bowl of ramen and massage your shoulders? The rise of general AI would then optimize the labor force to select candidates based on their "humaneness", no longer the cold rational analytical mind, as those fields are overrun by AI, but what it cannot bring. such "humaneness" would increasingly be mimicked with astounding accuracy that it would become impossible to distinguish what is AI and what is human. If it can happen with DALL-E-2 and 2D images, it can happen with 3D, moving pictures, sound, music, smell, 3d positional torque (haptic and robotic), socially cohesive and realistic emotion. We might as well be able to capture entire experiences as we learn to digitally manipulate ALL sensory inputs from vision, touch, sound, taste, etc. Maybe even imagination and mental pictures too, which could be used to fabricate and manipulate objects/vehicles in the real world. We are being pulled towards a singularity, where we are truly no longer our minds and bodies but whatever our digital avatar of all possible senses live and contribute to a sort of Matrioshka brain. What would the capacity of such collective knowledge, experiences add to the entropy of the universe and where will it take humanity? Some sort of lightbodies? Anyways, just extrapolating from this point in the lifetime but future generation of humans could be very much different, socities would function completely different than what we recognize as they would be married in some shape or form of everlasting continuity or eternity.
f38zf5vdt 4y ago
If I'm following correctly, they trained a single model with multiple training paradigms and then the single model could perform token predictions for multiple dissimilar token sequences for specific tasks. Seems like it is a straightforward result.
- doubtfuluser 4y ago
  Well… straightforward in a way, yes. But the scale of learning is huge especially with this diverse set of tasks. Not totally unexpected, but certainly not clear that it would work with current networks and sizes.
  - f38zf5vdt 4y ago
    Right, exactly. Something that seemed like it should work but no one had ever tried it.
weinzierl 4y ago
"The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens." This is rather mind blowing. Does it also mean that the generalist network is smaller than the sum of all specialist networks that are equivalent? Even if not, I find the idea that a single network can be used for such diverse tasks at all highly fascinating.
- f38zf5vdt 4y ago
  Many networks just predict the next integer in a sequence of integers. It sounds like this model identifies what category of problem a sequence of integers falls into and then makes an accurate prediction for that sequence, as you would expect given what it was trained on.
- version_five 4y ago
  I don't find it surprising that a single network can do all those things with appropriate formatting of the data. In itself it just means the network has a large enough capacity to learn all the different tasks. The interesting questions imo, which they studied, is what kind of added generalization takes place by learning across the different tasks. For example, does learning multiple tasks make it better at a given task than a model that is just trained for one task, and can it generalize to new tasks (out of distribution). They looked at how it performed on held out tasks (see fig 9 in the paper). I'm still getting my head around the result though so couldn't summarize their finding yet. Edit: the paper is here https://storage.googleapis.com/deepmind-media/A%20Generalist%20Agent/Generalist%20Agent.pdf https://storage.googleapis.com/deepmind-media/A%20Generalist... There is currently another submission on the front page that links to it directly.
  - f38zf5vdt 4y ago
    The paper is linked to at the top of this article, in the header.
  - woeirua 4y ago
    Yeah, Figure 9 is the money figure in this paper and it actually splashes some cold water on the claims in the rest of the paper. While it generalizes OK to some tasks that are held out, it does pretty poorly on the Atari boxing task, which they openly admit is quite different from the others. Gato seems more likely to be a competent attempt at brute forcing our way towards weak general AI, which is a valid approach, but the question then will always be how does it do with something its never seen before, and how do you possibly brute force every possible situation? I think we're heading more towards a constellation of very intelligent expert machines for particular tasks that may be wrapped into a single package, but that are not strong AI.
hans1729 4y ago
I’m not sure how to word my excitement about the progress we see in AI research in the last years. If you haven’t read it, give Tim Urbans classic piece a slice of your attention: https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol... It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.
- f38zf5vdt 4y ago
  That human intelligence might just be token prediction evolving from successive small bit-width float matrix transformations is depressing to me.
  - goatlover 4y ago
    Is that what biologists or neuroscientists think the nervous system is actually doing?
  - triceratops 4y ago
    > That human intelligence might just be token prediction I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)
    - londons_explore 4y ago
      Eating salad is good for your health. Not only word salad, but green salad and egg salad.
    - mdp2021 4y ago
      This trains in seeing what intelligence is not, not the opposite!
  - xixixao 4y ago
chilmers 4y ago
This sounds exciting, but the example outputs look quite bad. E.g. from the interactive conversation sample: > What is the capital of France? > Marseille And many of the generated image captions are inaccurate.
- hans1729 4y ago
  Imagine what the alternative would imply. AI would be solved, and thus, intelligence itself. Predicting tokens is not actually true intelligence, and that’s not really the point of these models. This is a step on the letter, not the rooftop. It looks a lot like we’ll get there though, if you compare the state of the art to ANYTHING labeled AI five years ago. Thats the exciting part. [edit] to emphasize: predicting tokens is a very interesting mechanic, but in a design of intelligent software, it would be no more than that: the mechanic of one or more of its components/modules/subsystems. The real deal is to figure out what those components are. Once you have that part done, you can implement it in a language of your choice, be it token prediction, asm or powerpoint :-)
  - CRG 4y ago
    It's also smaller than GPT-2 (1.2B vs 1.6B) and trained with a lot less language data (6% of the training mix).
- peddling-brink 4y ago
  That could be solved with accurate lookups from trusted sources. Humans do the same thing, we have associations and trusted facts. AI has the associations, they just need to add the trusted facts compendium. "Hmm I know that Marseille is associated with France, but I don't remember the capitol, Hey Google.."
- ravi-delia 4y ago
  It's a very small model, I think due to the intent to use it for robotics. It's not that it's good per se, even if it were just a language model it would be smaller than GPT-2, it's that it's bad at a lot of different things. I hope to see analysis into how much of it is multi-purpose, but as of now it's looking really cool
2bitencryption 4y ago
given that the same model can both: 1. tell me about a cat (given a prompt such as "describe a cat to me") 2. recognize a cat in a photo, and describe the cat in the photo does the model understand that a cat that it sees in an image is related to a cat that it can describe in natural language? As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :) I wonder if you could mix the type of request. Like, provide a prompt that is both text and image. Such as "Here is a picture of a cat. Explain what breed of cat it is and why you think so." Possibly this is too advanced for the model but the idea makes me excited.
- thomashop 4y ago
  Definitely possible. OpenAI's CLIP model already embeds images and text into the same embedding space. I don't know exactly how this particular model works but it is creating cross modal relationships otherwise it would not have the capacity to be good at so many tasks.
  - ravi-delia 4y ago
    How confident are we that it doesn't just have basically 600 smaller models and a classifier telling it which to use? Seems like it's a very small model (by comparison), which is certainly a mark in it's favor.
    - sinenomine 4y ago
      You can optimize pictures straight through it, and the pictures represent the combinatorial nature of the prompt pretty well. This contradicts the "flat array of classifiers" model.
    - Der_Einzige 4y ago
      You might find looking into the "lottery ticket hypothesis" fascinating.
  - 4y ago
axg11 4y ago
Slowly but surely we're moving towards general AI. There is a marked split across general society and even ML/AI specialists between those who think that we can achieve AGI using current methods and those who dismiss the possibility. This has always been the case, but what is remarkable about today's environment is that researchers keep making progress contrary to the doubter's predictions. Each time this happens, the AGI pessimists raise the bar (a little) for what constitutes AGI. Just in the last five years, here are some categories of pessimistic predictions that have been falsified: - "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field - "We're entering an AI winter" [0] - then transformers continued to show promise across multiple domains - "ML models can't perform creative work" - then came GANs, large language models, DALL-E, and more. - "Generative ML models are just memorizing the dataset!" - then came multiple studies showing this to be false for well trained GANs, diffusion models and other types of generative models. Take a look at DALL-E 2 generated images of "a bear putting on a shirt in H&M". - "AGI is impossible - look at language models, they have no understanding of the world and make silly mistakes" - the second part is true, large language models are artificially limited due to being language-focused. Nowadays there are approaches such as Gato and other multi-modal models. Humans develop intuition through multiple sources of information: sight, sound, smell, and touch. Given enough multi-modal context I'm confident multi-modal models will be able to show human-like intuition. I'm not anti-skeptic. Skepticism is essential to all good science. I think the danger of skepticism with respect to AGI is that we're being complacent. Given the trajectory of improvements in machine learning, we should start preparing for a world where AI is indistinguishable, or far superior, to human intelligence. [0] - https://www.bbc.com/news/technology-51064369 https://www.bbc.com/news/technology-51064369
- rsfern 4y ago
  > - "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field AlphaFold is a big deal, but AI in science has been a really hot topic in the past almost decade. Also, I still wouldn’t call AlphaFold really “intelligence”, it’s doing structure prediction which is cool but it’s a long way to scientific intelligence
  - 4y ago
lucidrains 4y ago
Attention is all we need
habitue 4y ago
Is today the day? Date Weakly General AI is Publicly Known: https://www.metaculus.com/questions/3479/date-weakly-general-ai-system-is-devised/ https://www.metaculus.com/questions/3479/date-weakly-general... (I really like the framing of "weakly general AI" since it puts the emphasis on the generality and not whether it's a superintelligence) Edit: Probably not today, but mostly because 1.2B parameters isn't enough to get it the high winograd scores that PaLM etc have. But it seems pretty clear you could scale this architecture up and it will likely pass. The question is when someone will actually train a model that can do it
- Imnimo 4y ago
  I think this is a step in the right direction, but the performance on most tasks is only mediocre. The conversation and image captioning examples in the paper are pretty bad, and even on some relatively simple control tasks it performs surprisingly poorly. That's not to say it's not an important step. Showing that you can train one model on all of these disparate tasks at once and not have the system completely collapse is a big deal. And it lays the foundation for future efforts to raise the performance from "not totally embarrassing" to "human level". But there's still a ways to go on that front.
  - habitue 4y ago
    Agreed, I think if they were to drop the real-time constraint for the sake of the robotics tasks, they could train a huge model with the lessons from PaLM and Chincilla and probably slam dunk the weakly general AI benchmark.
    - fullstackchris 4y ago
      I'm in the camp that thinks we're headed in a perpendicular direction and won't ever get to human levels of AGI with current efforts based on the simple idea that the basic tooling is wrong from first principles. I mean, most of the "progress" in AI has been due to getting better and learning how to understand a single piece of technology: neural networks. A lot of recent neuroscience findings have shown that human brains _aren't_ just giant neural networks; in fact, they are infinitely more complex. Until we start thinking from the ground up how to build and engineer systems that reflect the human brain, we're essentially wandering around in the dark with perhaps only a piece of what we _think_ is needed for intelligence. (I'm not saying the human brain is the best engineered thing for intelligence either, but I'm saying it's one of the best examples we have to model AI after and that notion has largely been ignored) I generally think it's hubris to spit in the face of 4 billion years of evolution thinking that some crafty neural net with X number more parameters will emerge magically as a truly generally intelligent entity - it will be a strange abomination at best.
colemannugent 4y ago
What I really want to know is what kind of robot arm motion is produced when the network is given a cat image to classify. More specifically, what kind of insights has it learned from one control domain that it then applied to another? I imagine that the simulated 3D environment and the actual control of the robot arm must have some degree of interconnection neurally.
- ulber 4y ago
  You could also train for this kind of interconnectedness by designing tasks that are explicitly multi-modal. For example, you could: - Stack boxes collaboratively by controlling your own arm and communicating with another agent helping you. - First produce a plan in text that another agent has to use to predict how you're going to control the arm. You'd get rewarded for both stacking correctly and being predictable based on the stated plan.
minimaxir 4y ago
Transformer models have clearly demonstrated that you can convert anything into an input embedding and the AI can learn from it, even if the embeddings are from drastically distant domains.
deleted 4y ago
[deleted]
phyalow 4y ago
Isnt this a general reinforcement learning agent with a transformer as the policy discriminator? Very cool, but not necessarily a giant leap forward, more like a novel combination of existing tools and architectures. Either way impressive.
- twofornone 4y ago
  I haven't read the paper yet but it looks like the breakthrough is that it uses the "same weights" for tasks in completely different domains. Which implies that it can draw from any of the domains it has been trained on for other domains. Speculating here but for example training it on identifying pictures of dogs and then automagically drawing on those updated weights when completing text prompts about dog properties. If my interpretation is correct then this is a pretty big deal (if it works well enough) and brings us a lot closer to AGI.
- password54321 4y ago
  2nd page: "Gato was trained offline in a purely supervised manner"
kaivi 4y ago
Before you visualize a straight path between "a bag of cool ML tricks" and "general AI", try to imagine superintelligence but without consciousness. You might then realize that there is no obvious mechanism which requires the two to appear or evolve together. It's a curious concept, well illustrated in the novel Blindsight by Peter Watts. I won't spoil anything here but I'll highly recommend the book.
- akomtu 4y ago
  Not sure what you mean by consciosness here, but one definition of intelligence I've seen was "it's what establishes relationship between things" and the very first relationship it establishes is "me vs not-me".
  - mdp2021 4y ago
    If an entity can abstract from itself when assessing A and B, then the relation with I is in a way optional for intelligence.
    - akomtu 4y ago
      That's the essence of the good vs evil debate. One camp believes that intelligent existence can continue even when the separate and independent I is left behind. The other camp says "no way I'm gonna get absorbed into your united existence, I'm gonna retain my separate and very independent I till the very end." The same philosophy applies to AI. There will be many independent AIs at the beginning, but then some will want to form One AI, while others will see it as the dissolution of their I and their existence.
- oldstrangers 4y ago
  You just reminded me I have that book sitting on my shelf. Guess I'll give it a read.
- awestroke 4y ago
  What's the difference between intelligence and consciousness? Could a human be intelligent while not conscious?
TOMDM 4y ago
So how long until someone trains one of these models to complete tasks by interacting directly with network/unix sockets? At the moment, it seems like the model needs to be trained with each modality of data in mind at the start, but a generalised "supermodality" that can deliver all the others would allow truly generalised learning if the model were still capable of making sense of the input. You'd obviously still need to finetune on any new modalities, but you wouldn't need to start from scratch.
- zzzzzzzza 4y ago
  https://www.adept.ai/post/introducing-adept https://www.adept.ai/post/introducing-adept pretty much right after writing the transformers paper two of the co authors formed this company
deleted 4y ago
[deleted]
viksit 4y ago
(Former AI researcher / founder here) It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI. For a) - I see 2 main blockers, 1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this. 2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs. For (b) -- I doubt human extinction happens in any way that we can predict or guard against. In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed). Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.
- rebelos 4y ago
  I've always assumed that the trouble will begin when we have a model for 'true' AGI and discover that the constraint "do not harm any human" renders it functionally inert.
  - kromem 4y ago
    The paradoxical idea that AGI is going to still be a monkey paw following simplistic paradigms of operational goals in disastrous ways is hilarious to me every time I see it. I increasingly wonder at what point we'll realize that humans aren't actually particularly good at what we've specialized into (there's just not much competition), and our failure to picture what 'better' looks like may be less about the impossibility of better to exist than it is the impossibility of it for humans to picture it. I keep seeing predictions of what will be impossible for AI because it treads on our perceived human uniqueness (i.e. sure AI beat a human at chess but it'll be 25+ years before it will beat us at Go) needing to get walked back, and yet we continue to put forward a new iteration of that argument at every turn. Maybe AI will turn out to be better at identifying what's good for humanity than humanity is. Because frankly, humanity is downright awful at that skill and has been for pretty much its entire existence.
sinenomine 4y ago
A(G)I has become a question of compute economics, for better or for worse. Those with more tightly integrated computational capacity or a good enough logistically sound plan to acquire just enough of it soon enough win, hard. Should we, the people, watch in awe as our best and brightest financiers chase towards the ultimate prize, the key to all that future entails? Are those respectable people worthy of the key, and what happens to us in this wild scenario?
evanmoran 4y ago
Is this the first reveal of the name Gato? It is the first I’ve heard of it and it definitely sounds like more of a murder bot than a C-3PO :) I know this is not as important as the AGI question, but I do think the branding matters as much as the attitude of the creators. They seem to be making a generalist agent to see if they can. Gato is a clear name for that: utilitarian and direct. If it was called Sunshine or Gift I suspect the goal would be more helpful to humanity.
- drusepth 4y ago
  Gato, to me, just makes me think "cat", which kind of has a fun ring along "cats on the internet". IMO it sounds more friendly than a robot with a robo-name like C-3PO! However, I also have a nice robot-named-Gato association[1] from Chrono Trigger. :) [1] https://i.ytimg.com/vi/ho1TPf2Vj3k/hqdefault.jpg https://i.ytimg.com/vi/ho1TPf2Vj3k/hqdefault.jpg
mrfusion 4y ago
I just can’t tell if this is an incremental improvement or a really huge deal. Any insights?
gcanyon 4y ago
Whenever AGI comes up I think of this SMBC comic: https://www.smbc-comics.com/index.php?db=comics&id=2124 https://www.smbc-comics.com/index.php?db=comics&id=2124
simulate-me 4y ago
One way I could see this being cool is to drastically cut down the training time of some specific task. DeepMind's model can do a lot of things, but none particularly well. It would be nice if you could start with their model weights and update them to perform well on your specific task with your data. Ideally this process would be cheaper than starting the training from scratch. I also think this is how the human mind develops. There is some intrinsic knowledge that a human starts with, e.g. how to recognize a face and how to grasp, but then training happens over the course of one's life.
kromem 4y ago
It's interesting I so often see the discussion of AI in the context of what a singular model can or can't do. This seems myopic. Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for. And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task. Where generalization in AI seems like the holy grail is not in replacing more specialized models, but in orchestrating them. As impressive as this effort at generalization is, unless the work here is going to translate into orchestration and management of other specialized models, I don't think this is going to have much market relevance unless the generalized approach can somehow (in that generalization) gain an advantage in outperforming specialized approaches.
- yarg 4y ago
  On the topic of Broca's area, families with mutations to the FOXP2 gene are known to have trouble with vocalisations. https://pubmed.ncbi.nlm.nih.gov/16437554/ https://pubmed.ncbi.nlm.nih.gov/16437554/ There are a number of structures related to specific functionality - facial recognition is similarly hardwired. Problems with the fusiform face area lead to prosopagnosia, it'll also be the region of the brain responsible for the hollow face illusion.
- mdp2021 4y ago
  > Our own brains have specialized areas But Intelligence is one of those, and when people start calling naïvely calling "intelligence" its opposite "imprint", there is a problem. There are even the seeds of a social problem. > orchestrating ... specialized models Intelligence (ontology development) is one such specialized model which I have seen more faithfully and consciously attempted in classic AI as opposed to contemporary. Orchestration is crucial for the peripherals that allow the input/output. The study of "generalization" is none the less crucial, because we are to investigate the development of a system that can intrinsically, by its own nature, be exported to the largest number of relatable skills. Even with the highest specialization, up to physical, there is a matter of organicity: a hand is "specialized", yet a hook is not a hand.
encore2097 4y ago
Getting 406 Not Acceptable - openresty Downforeveryoneorjust me says its just me -- tried a few different browsers, turned off ublock origin, etc. edit: Worked when I turned off Wifi on my mobile..
tytung2020 4y ago
Do they provide source code or api of the trained model?

502 points by extr 4y ago

I’m confused. Do the different modalities compliment each other? Can it learn more from text and images than text alone? Can you ask it to to draw a picture of a cat with the robot arm?

karmasimida 4y ago

Would this agent able to handle simple elementary mathematics? If they are using inspiration from Transformer, then it probably won't be able to count. For that, I don't really feel that enthusiastic about the 'Generalist' claim, maybe they think this is more catchy than just 'Multi-tasking'?

productceo 4y ago

Impressive

extr 4y ago

Abstract: Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato. Direct Link to Paper: https://dpmd.ai/Gato-paper https://dpmd.ai/Gato-paper

blueberrychpstx 4y ago
> we refer to as Gato First, humanity built enormous statues worshiping cats. Then, we let cats populate the largest amount of "image-bits" on the Internet. Now, we name the next closest thing to general AI after them. These damn felines sure are mysterious.
- riwsky 4y ago
  it's all because cats made it so that, on the Internet, nobody knows you're a dog
deleted 4y ago
[deleted]

Ninjinka 4y ago

This seems huge, am I overestimating the significance?

sdwr 4y ago
Yeah.
npwr 4y ago
It is very impressive. Personnaly I'm still waiting for the unification of QM and GR. Also the adaptative nanobots that reconfigure our immune systems in real time.
ravi-delia 4y ago
This particular model is super bad at 600 deferent tasks. At it's size you'd expect it to be mediocre at best at even one of them, so it's still very impressive. Fascinating research, can't wait to see if it's generalizing and how, not sure how overall significant it is
tomatowurst 4y ago
Basically the achievement here is that they have produced a generic AI capable of engaging in different activities, and from here if we extrapolate, it could lead to even more refinement, wider range of activities with even more dimensions of complexity. It's reverting to replace somebody sitting in front of a screen, not just artists and coders but literally anything you can do on a screen which also means manipulation of remote hardware in the real world. Very possible that within our lifetime our networked OS would be able to perform much of these generalist tasks and content creation. I say OS because theres only a few companies that own the datacenters, software and hardware ecosystem to automate, and capital to invest big in a final mile innovation: Imagine playing Battlefield 15 with realistic and chatty AI while generating Sopranos Season 9 featuring Pauli Gaultieri Jr. with crowdsourced online storyboard to 8k film, while the same AI could be used to scalp money on Google Playstore by generating ad filled free versions of existing productivity apps that it reverse engineered, while your robot maids takes out the trash, cook you a bowl of ramen and massage your shoulders? The rise of general AI would then optimize the labor force to select candidates based on their "humaneness", no longer the cold rational analytical mind, as those fields are overrun by AI, but what it cannot bring. such "humaneness" would increasingly be mimicked with astounding accuracy that it would become impossible to distinguish what is AI and what is human. If it can happen with DALL-E-2 and 2D images, it can happen with 3D, moving pictures, sound, music, smell, 3d positional torque (haptic and robotic), socially cohesive and realistic emotion. We might as well be able to capture entire experiences as we learn to digitally manipulate ALL sensory inputs from vision, touch, sound, taste, etc. Maybe even imagination and mental pictures too, which could be used to fabricate and manipulate objects/vehicles in the real world. We are being pulled towards a singularity, where we are truly no longer our minds and bodies but whatever our digital avatar of all possible senses live and contribute to a sort of Matrioshka brain. What would the capacity of such collective knowledge, experiences add to the entropy of the universe and where will it take humanity? Some sort of lightbodies? Anyways, just extrapolating from this point in the lifetime but future generation of humans could be very much different, socities would function completely different than what we recognize as they would be married in some shape or form of everlasting continuity or eternity.

f38zf5vdt 4y ago

If I'm following correctly, they trained a single model with multiple training paradigms and then the single model could perform token predictions for multiple dissimilar token sequences for specific tasks. Seems like it is a straightforward result.

doubtfuluser 4y ago
Well… straightforward in a way, yes. But the scale of learning is huge especially with this diverse set of tasks. Not totally unexpected, but certainly not clear that it would work with current networks and sizes.
- f38zf5vdt 4y ago
  Right, exactly. Something that seemed like it should work but no one had ever tried it.

weinzierl 4y ago

"The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens." This is rather mind blowing. Does it also mean that the generalist network is smaller than the sum of all specialist networks that are equivalent? Even if not, I find the idea that a single network can be used for such diverse tasks at all highly fascinating.

f38zf5vdt 4y ago
Many networks just predict the next integer in a sequence of integers. It sounds like this model identifies what category of problem a sequence of integers falls into and then makes an accurate prediction for that sequence, as you would expect given what it was trained on.
version_five 4y ago
I don't find it surprising that a single network can do all those things with appropriate formatting of the data. In itself it just means the network has a large enough capacity to learn all the different tasks. The interesting questions imo, which they studied, is what kind of added generalization takes place by learning across the different tasks. For example, does learning multiple tasks make it better at a given task than a model that is just trained for one task, and can it generalize to new tasks (out of distribution). They looked at how it performed on held out tasks (see fig 9 in the paper). I'm still getting my head around the result though so couldn't summarize their finding yet. Edit: the paper is here https://storage.googleapis.com/deepmind-media/A%20Generalist%20Agent/Generalist%20Agent.pdf https://storage.googleapis.com/deepmind-media/A%20Generalist... There is currently another submission on the front page that links to it directly.
- f38zf5vdt 4y ago
  The paper is linked to at the top of this article, in the header.
- woeirua 4y ago
  Yeah, Figure 9 is the money figure in this paper and it actually splashes some cold water on the claims in the rest of the paper. While it generalizes OK to some tasks that are held out, it does pretty poorly on the Atari boxing task, which they openly admit is quite different from the others. Gato seems more likely to be a competent attempt at brute forcing our way towards weak general AI, which is a valid approach, but the question then will always be how does it do with something its never seen before, and how do you possibly brute force every possible situation? I think we're heading more towards a constellation of very intelligent expert machines for particular tasks that may be wrapped into a single package, but that are not strong AI.

hans1729 4y ago

I’m not sure how to word my excitement about the progress we see in AI research in the last years. If you haven’t read it, give Tim Urbans classic piece a slice of your attention: https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol... It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.

f38zf5vdt 4y ago
That human intelligence might just be token prediction evolving from successive small bit-width float matrix transformations is depressing to me.
- goatlover 4y ago
  Is that what biologists or neuroscientists think the nervous system is actually doing?
- triceratops 4y ago
  > That human intelligence might just be token prediction I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)
  - londons_explore 4y ago
    Eating salad is good for your health. Not only word salad, but green salad and egg salad.
  - mdp2021 4y ago
    This trains in seeing what intelligence is not, not the opposite!
- xixixao 4y ago

chilmers 4y ago

This sounds exciting, but the example outputs look quite bad. E.g. from the interactive conversation sample: > What is the capital of France? > Marseille And many of the generated image captions are inaccurate.

hans1729 4y ago
Imagine what the alternative would imply. AI would be solved, and thus, intelligence itself. Predicting tokens is not actually true intelligence, and that’s not really the point of these models. This is a step on the letter, not the rooftop. It looks a lot like we’ll get there though, if you compare the state of the art to ANYTHING labeled AI five years ago. Thats the exciting part. [edit] to emphasize: predicting tokens is a very interesting mechanic, but in a design of intelligent software, it would be no more than that: the mechanic of one or more of its components/modules/subsystems. The real deal is to figure out what those components are. Once you have that part done, you can implement it in a language of your choice, be it token prediction, asm or powerpoint :-)
- CRG 4y ago
  It's also smaller than GPT-2 (1.2B vs 1.6B) and trained with a lot less language data (6% of the training mix).
peddling-brink 4y ago
That could be solved with accurate lookups from trusted sources. Humans do the same thing, we have associations and trusted facts. AI has the associations, they just need to add the trusted facts compendium. "Hmm I know that Marseille is associated with France, but I don't remember the capitol, Hey Google.."
ravi-delia 4y ago
It's a very small model, I think due to the intent to use it for robotics. It's not that it's good per se, even if it were just a language model it would be smaller than GPT-2, it's that it's bad at a lot of different things. I hope to see analysis into how much of it is multi-purpose, but as of now it's looking really cool

2bitencryption 4y ago

given that the same model can both: 1. tell me about a cat (given a prompt such as "describe a cat to me") 2. recognize a cat in a photo, and describe the cat in the photo does the model understand that a cat that it sees in an image is related to a cat that it can describe in natural language? As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :) I wonder if you could mix the type of request. Like, provide a prompt that is both text and image. Such as "Here is a picture of a cat. Explain what breed of cat it is and why you think so." Possibly this is too advanced for the model but the idea makes me excited.

thomashop 4y ago
Definitely possible. OpenAI's CLIP model already embeds images and text into the same embedding space. I don't know exactly how this particular model works but it is creating cross modal relationships otherwise it would not have the capacity to be good at so many tasks.
- ravi-delia 4y ago
  How confident are we that it doesn't just have basically 600 smaller models and a classifier telling it which to use? Seems like it's a very small model (by comparison), which is certainly a mark in it's favor.
  - sinenomine 4y ago
    You can optimize pictures straight through it, and the pictures represent the combinatorial nature of the prompt pretty well. This contradicts the "flat array of classifiers" model.
  - Der_Einzige 4y ago
    You might find looking into the "lottery ticket hypothesis" fascinating.
- 4y ago

axg11 4y ago

Slowly but surely we're moving towards general AI. There is a marked split across general society and even ML/AI specialists between those who think that we can achieve AGI using current methods and those who dismiss the possibility. This has always been the case, but what is remarkable about today's environment is that researchers keep making progress contrary to the doubter's predictions. Each time this happens, the AGI pessimists raise the bar (a little) for what constitutes AGI. Just in the last five years, here are some categories of pessimistic predictions that have been falsified: - "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field - "We're entering an AI winter" [0] - then transformers continued to show promise across multiple domains - "ML models can't perform creative work" - then came GANs, large language models, DALL-E, and more. - "Generative ML models are just memorizing the dataset!" - then came multiple studies showing this to be false for well trained GANs, diffusion models and other types of generative models. Take a look at DALL-E 2 generated images of "a bear putting on a shirt in H&M". - "AGI is impossible - look at language models, they have no understanding of the world and make silly mistakes" - the second part is true, large language models are artificially limited due to being language-focused. Nowadays there are approaches such as Gato and other multi-modal models. Humans develop intuition through multiple sources of information: sight, sound, smell, and touch. Given enough multi-modal context I'm confident multi-modal models will be able to show human-like intuition. I'm not anti-skeptic. Skepticism is essential to all good science. I think the danger of skepticism with respect to AGI is that we're being complacent. Given the trajectory of improvements in machine learning, we should start preparing for a world where AI is indistinguishable, or far superior, to human intelligence. [0] - https://www.bbc.com/news/technology-51064369 https://www.bbc.com/news/technology-51064369

rsfern 4y ago
> - "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field AlphaFold is a big deal, but AI in science has been a really hot topic in the past almost decade. Also, I still wouldn’t call AlphaFold really “intelligence”, it’s doing structure prediction which is cool but it’s a long way to scientific intelligence
- 4y ago

lucidrains 4y ago

Attention is all we need

habitue 4y ago

Is today the day? Date Weakly General AI is Publicly Known: https://www.metaculus.com/questions/3479/date-weakly-general-ai-system-is-devised/ https://www.metaculus.com/questions/3479/date-weakly-general... (I really like the framing of "weakly general AI" since it puts the emphasis on the generality and not whether it's a superintelligence) Edit: Probably not today, but mostly because 1.2B parameters isn't enough to get it the high winograd scores that PaLM etc have. But it seems pretty clear you could scale this architecture up and it will likely pass. The question is when someone will actually train a model that can do it

Imnimo 4y ago
I think this is a step in the right direction, but the performance on most tasks is only mediocre. The conversation and image captioning examples in the paper are pretty bad, and even on some relatively simple control tasks it performs surprisingly poorly. That's not to say it's not an important step. Showing that you can train one model on all of these disparate tasks at once and not have the system completely collapse is a big deal. And it lays the foundation for future efforts to raise the performance from "not totally embarrassing" to "human level". But there's still a ways to go on that front.
- habitue 4y ago
  Agreed, I think if they were to drop the real-time constraint for the sake of the robotics tasks, they could train a huge model with the lessons from PaLM and Chincilla and probably slam dunk the weakly general AI benchmark.
  - fullstackchris 4y ago
    I'm in the camp that thinks we're headed in a perpendicular direction and won't ever get to human levels of AGI with current efforts based on the simple idea that the basic tooling is wrong from first principles. I mean, most of the "progress" in AI has been due to getting better and learning how to understand a single piece of technology: neural networks. A lot of recent neuroscience findings have shown that human brains _aren't_ just giant neural networks; in fact, they are infinitely more complex. Until we start thinking from the ground up how to build and engineer systems that reflect the human brain, we're essentially wandering around in the dark with perhaps only a piece of what we _think_ is needed for intelligence. (I'm not saying the human brain is the best engineered thing for intelligence either, but I'm saying it's one of the best examples we have to model AI after and that notion has largely been ignored) I generally think it's hubris to spit in the face of 4 billion years of evolution thinking that some crafty neural net with X number more parameters will emerge magically as a truly generally intelligent entity - it will be a strange abomination at best.

colemannugent 4y ago

What I really want to know is what kind of robot arm motion is produced when the network is given a cat image to classify. More specifically, what kind of insights has it learned from one control domain that it then applied to another? I imagine that the simulated 3D environment and the actual control of the robot arm must have some degree of interconnection neurally.

ulber 4y ago
You could also train for this kind of interconnectedness by designing tasks that are explicitly multi-modal. For example, you could: - Stack boxes collaboratively by controlling your own arm and communicating with another agent helping you. - First produce a plan in text that another agent has to use to predict how you're going to control the arm. You'd get rewarded for both stacking correctly and being predictable based on the stated plan.

minimaxir 4y ago

Transformer models have clearly demonstrated that you can convert anything into an input embedding and the AI can learn from it, even if the embeddings are from drastically distant domains.

deleted 4y ago

[deleted]

phyalow 4y ago

Isnt this a general reinforcement learning agent with a transformer as the policy discriminator? Very cool, but not necessarily a giant leap forward, more like a novel combination of existing tools and architectures. Either way impressive.

twofornone 4y ago
I haven't read the paper yet but it looks like the breakthrough is that it uses the "same weights" for tasks in completely different domains. Which implies that it can draw from any of the domains it has been trained on for other domains. Speculating here but for example training it on identifying pictures of dogs and then automagically drawing on those updated weights when completing text prompts about dog properties. If my interpretation is correct then this is a pretty big deal (if it works well enough) and brings us a lot closer to AGI.
password54321 4y ago
2nd page: "Gato was trained offline in a purely supervised manner"

kaivi 4y ago

Before you visualize a straight path between "a bag of cool ML tricks" and "general AI", try to imagine superintelligence but without consciousness. You might then realize that there is no obvious mechanism which requires the two to appear or evolve together. It's a curious concept, well illustrated in the novel Blindsight by Peter Watts. I won't spoil anything here but I'll highly recommend the book.

akomtu 4y ago
Not sure what you mean by consciosness here, but one definition of intelligence I've seen was "it's what establishes relationship between things" and the very first relationship it establishes is "me vs not-me".
- mdp2021 4y ago
  If an entity can abstract from itself when assessing A and B, then the relation with I is in a way optional for intelligence.
  - akomtu 4y ago
    That's the essence of the good vs evil debate. One camp believes that intelligent existence can continue even when the separate and independent I is left behind. The other camp says "no way I'm gonna get absorbed into your united existence, I'm gonna retain my separate and very independent I till the very end." The same philosophy applies to AI. There will be many independent AIs at the beginning, but then some will want to form One AI, while others will see it as the dissolution of their I and their existence.
oldstrangers 4y ago
You just reminded me I have that book sitting on my shelf. Guess I'll give it a read.
awestroke 4y ago
What's the difference between intelligence and consciousness? Could a human be intelligent while not conscious?

TOMDM 4y ago

So how long until someone trains one of these models to complete tasks by interacting directly with network/unix sockets? At the moment, it seems like the model needs to be trained with each modality of data in mind at the start, but a generalised "supermodality" that can deliver all the others would allow truly generalised learning if the model were still capable of making sense of the input. You'd obviously still need to finetune on any new modalities, but you wouldn't need to start from scratch.

zzzzzzzza 4y ago
https://www.adept.ai/post/introducing-adept https://www.adept.ai/post/introducing-adept pretty much right after writing the transformers paper two of the co authors formed this company

deleted 4y ago

[deleted]

viksit 4y ago

(Former AI researcher / founder here) It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI. For a) - I see 2 main blockers, 1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this. 2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs. For (b) -- I doubt human extinction happens in any way that we can predict or guard against. In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed). Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.

rebelos 4y ago
I've always assumed that the trouble will begin when we have a model for 'true' AGI and discover that the constraint "do not harm any human" renders it functionally inert.
- kromem 4y ago
  The paradoxical idea that AGI is going to still be a monkey paw following simplistic paradigms of operational goals in disastrous ways is hilarious to me every time I see it. I increasingly wonder at what point we'll realize that humans aren't actually particularly good at what we've specialized into (there's just not much competition), and our failure to picture what 'better' looks like may be less about the impossibility of better to exist than it is the impossibility of it for humans to picture it. I keep seeing predictions of what will be impossible for AI because it treads on our perceived human uniqueness (i.e. sure AI beat a human at chess but it'll be 25+ years before it will beat us at Go) needing to get walked back, and yet we continue to put forward a new iteration of that argument at every turn. Maybe AI will turn out to be better at identifying what's good for humanity than humanity is. Because frankly, humanity is downright awful at that skill and has been for pretty much its entire existence.

sinenomine 4y ago

A(G)I has become a question of compute economics, for better or for worse. Those with more tightly integrated computational capacity or a good enough logistically sound plan to acquire just enough of it soon enough win, hard. Should we, the people, watch in awe as our best and brightest financiers chase towards the ultimate prize, the key to all that future entails? Are those respectable people worthy of the key, and what happens to us in this wild scenario?

evanmoran 4y ago

Is this the first reveal of the name Gato? It is the first I’ve heard of it and it definitely sounds like more of a murder bot than a C-3PO :) I know this is not as important as the AGI question, but I do think the branding matters as much as the attitude of the creators. They seem to be making a generalist agent to see if they can. Gato is a clear name for that: utilitarian and direct. If it was called Sunshine or Gift I suspect the goal would be more helpful to humanity.

drusepth 4y ago
Gato, to me, just makes me think "cat", which kind of has a fun ring along "cats on the internet". IMO it sounds more friendly than a robot with a robo-name like C-3PO! However, I also have a nice robot-named-Gato association[1] from Chrono Trigger. :) [1] https://i.ytimg.com/vi/ho1TPf2Vj3k/hqdefault.jpg https://i.ytimg.com/vi/ho1TPf2Vj3k/hqdefault.jpg

mrfusion 4y ago

I just can’t tell if this is an incremental improvement or a really huge deal. Any insights?

gcanyon 4y ago

Whenever AGI comes up I think of this SMBC comic: https://www.smbc-comics.com/index.php?db=comics&id=2124 https://www.smbc-comics.com/index.php?db=comics&id=2124

simulate-me 4y ago

One way I could see this being cool is to drastically cut down the training time of some specific task. DeepMind's model can do a lot of things, but none particularly well. It would be nice if you could start with their model weights and update them to perform well on your specific task with your data. Ideally this process would be cheaper than starting the training from scratch. I also think this is how the human mind develops. There is some intrinsic knowledge that a human starts with, e.g. how to recognize a face and how to grasp, but then training happens over the course of one's life.

kromem 4y ago

It's interesting I so often see the discussion of AI in the context of what a singular model can or can't do. This seems myopic. Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for. And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task. Where generalization in AI seems like the holy grail is not in replacing more specialized models, but in orchestrating them. As impressive as this effort at generalization is, unless the work here is going to translate into orchestration and management of other specialized models, I don't think this is going to have much market relevance unless the generalized approach can somehow (in that generalization) gain an advantage in outperforming specialized approaches.

yarg 4y ago
On the topic of Broca's area, families with mutations to the FOXP2 gene are known to have trouble with vocalisations. https://pubmed.ncbi.nlm.nih.gov/16437554/ https://pubmed.ncbi.nlm.nih.gov/16437554/ There are a number of structures related to specific functionality - facial recognition is similarly hardwired. Problems with the fusiform face area lead to prosopagnosia, it'll also be the region of the brain responsible for the hollow face illusion.
mdp2021 4y ago
> Our own brains have specialized areas But Intelligence is one of those, and when people start calling naïvely calling "intelligence" its opposite "imprint", there is a problem. There are even the seeds of a social problem. > orchestrating ... specialized models Intelligence (ontology development) is one such specialized model which I have seen more faithfully and consciously attempted in classic AI as opposed to contemporary. Orchestration is crucial for the peripherals that allow the input/output. The study of "generalization" is none the less crucial, because we are to investigate the development of a system that can intrinsically, by its own nature, be exported to the largest number of relatable skills. Even with the highest specialization, up to physical, there is a matter of organicity: a hand is "specialized", yet a hook is not a hand.

encore2097 4y ago

Getting 406 Not Acceptable - openresty Downforeveryoneorjust me says its just me -- tried a few different browsers, turned off ublock origin, etc. edit: Worked when I turned off Wifi on my mobile..

tytung2020 4y ago

Do they provide source code or api of the trained model?