4 ms·

I don't think this is THAT world changing. Generalist / multitask learning is already displayed in many models across orgs like Deepmind and others. This Graph

I don't think this is THAT world changing. Generalist / multitask learning is already displayed in many models across orgs like Deepmind and others. This Graph NN model is performing algorithms that typically rely on graph theory. It's unsurprising that there's little friction and high success in combining embedding spaces on various graph algorithms by representing those in a graph. If this model was realizing special success on other domains it would seem more significant.

Jensson 4y ago
The interesting part would be if they could put this somewhere on the path of a large language model, so that it could learn to apply logic to its transformations instead of just symbolically manipulate things. Then maybe we could get a language model that can do math.
- blovescoffee 4y ago
  Ha well that _would_ be interesting. This work is interesting on its own BUT yeah it doesn't yet do what you mentioned and, to me, its conclusions are not that surprising.
- thomasahle 4y ago
  The question is whether this is a better approach than just giving the LLM access to calculator / python-interpreter.
  - IIAOPSW 4y ago
    Whats the difference between math and a language? Its all just literals (numbers) and verbs relating them (+,-,*,/).
    - thomasahle 4y ago
      Large Language Models are bad at a lot of the same things humans are bad at. For example, humans can't add or multiply large numbers in their head. They need a piece of paper as an axillary memory. Or a calculator. If we need to find the shortest path in a graph, we write a short program to do it. Likewise a Large Language Model can easily write the source code to do addition for arbitrarily large numbers, or solve graph related problems. (See AlphaCode.) It's not clear that they need to be "Generalist Algorithmic Learners" as this paper suggests. Time will tell.
  - kelseyfrog 4y ago
    Or a little recurrent ALU side-chain glued on. It would at the very least put a stop to the "It can't even multiple two four-digit numbers correctly," squad which I admit, I can't do in my head either.

Jensson 4y ago

The interesting part would be if they could put this somewhere on the path of a large language model, so that it could learn to apply logic to its transformations instead of just symbolically manipulate things. Then maybe we could get a language model that can do math.

blovescoffee 4y ago
Ha well that _would_ be interesting. This work is interesting on its own BUT yeah it doesn't yet do what you mentioned and, to me, its conclusions are not that surprising.
thomasahle 4y ago
The question is whether this is a better approach than just giving the LLM access to calculator / python-interpreter.
- IIAOPSW 4y ago
  Whats the difference between math and a language? Its all just literals (numbers) and verbs relating them (+,-,*,/).
  - thomasahle 4y ago
    Large Language Models are bad at a lot of the same things humans are bad at. For example, humans can't add or multiply large numbers in their head. They need a piece of paper as an axillary memory. Or a calculator. If we need to find the shortest path in a graph, we write a short program to do it. Likewise a Large Language Model can easily write the source code to do addition for arbitrarily large numbers, or solve graph related problems. (See AlphaCode.) It's not clear that they need to be "Generalist Algorithmic Learners" as this paper suggests. Time will tell.
- kelseyfrog 4y ago
  Or a little recurrent ALU side-chain glued on. It would at the very least put a stop to the "It can't even multiple two four-digit numbers correctly," squad which I admit, I can't do in my head either.