5 ms·
A Generalist Neural Algorithmic Learner
- deleted 4y ago[deleted]
- yamrzou 4y agoThis is from DeepMind. Twitter thread: https://twitter.com/DeepMind/status/1600852768125726720 https://twitter.com/DeepMind/status/1600852768125726720
- blovescoffee 4y agoI don't think this is THAT world changing. Generalist / multitask learning is already displayed in many models across orgs like Deepmind and others. This Graph NN model is performing algorithms that typically rely on graph theory. It's unsurprising that there's little friction and high success in combining embedding spaces on various graph algorithms by representing those in a graph. If this model was realizing special success on other domains it would seem more significant.
- Jensson 4y agoThe interesting part would be if they could put this somewhere on the path of a large language model, so that it could learn to apply logic to its transformations instead of just symbolically manipulate things. Then maybe we could get a language model that can do math.
- blovescoffee 4y agoHa well that _would_ be interesting. This work is interesting on its own BUT yeah it doesn't yet do what you mentioned and, to me, its conclusions are not that surprising.
- thomasahle 4y agoThe question is whether this is a better approach than just giving the LLM access to calculator / python-interpreter.
- IIAOPSW 4y agoWhats the difference between math and a language? Its all just literals (numbers) and verbs relating them (+,-,*,/).
- thomasahle 4y agoLarge Language Models are bad at a lot of the same things humans are bad at. For example, humans can't add or multiply large numbers in their head. They need a piece of paper as an axillary memory. Or a calculator. If we need to find the shortest path in a graph, we write a short program to do it. Likewise a Large Language Model can easily write the source code to do addition for arbitrarily large numbers, or solve graph related problems. (See AlphaCode.) It's not clear that they need to be "Generalist Algorithmic Learners" as this paper suggests. Time will tell.
- kelseyfrog 4y agoOr a little recurrent ALU side-chain glued on. It would at the very least put a stop to the "It can't even multiple two four-digit numbers correctly," squad which I admit, I can't do in my head either.
- stephc_int13 4y agoThis could be a piece of the AGI puzzle. Learning how to combine/integrate various kinds of ANNs trained on vastly different domains is something that will have to be done at some point. At least this seems to be a reasonable path to explore.
- optimalsolver 4y agoIt feels like there's an interesting release from the one of the major AI labs nearly every week now. Is this a hype wave before the next AI winter, or are we accelerating towards something truly profound?
- AbrahamParangi 4y agoThe hype cycle wants to end, but the pace of technological advance is breaking it.
- riku_iki 4y agothere many hype like releases, with bold claims you can't verify, but very few cases when results are good enough to be applied on real world tasks.
- UncleOxidant 4y agoAre some of these labs under layoff pressure?
- joaquincabezas 4y agoCheck https://logconference.org/schedule-tutorials/#neural-algorithmic-reasoning https://logconference.org/schedule-tutorials/#neural-algorit... for a tutorial happening in less than 24 hours. LoG conference is free to attend and online :)
- IIAOPSW 4y ago
- joko42 4y agoMan the pace of AI progress these days is bonkers.