3 ms·

I think he's proposing that the LLM should know how to call out to a knowledge engine at inference time. He thinks the knowledge engine continues to be its own

I think he's proposing that the LLM should know how to call out to a knowledge engine at inference time. He thinks the knowledge engine continues to be its own (human-curated) system of knowledge that is valuable. I am suggesting the LLM will (effectively) call out to a knowledge engine at training time, learn everything the knowledge engine knows, and render it obsolete. So it's similar in some sense (collaboration between the two systems), but crucially, a diametrically opposed prediction in terms of the long-term viability of Wolfram Alpha. Crucially, he says "[an LLM] just isn’t a good fit in situations where there are structured computational things to do", but I think it's dubious to claim this; LLMs can learn structured domains too, if they are well-represented in the training set. edit to add: I see that you're specifically noting the LaMDA point, yes, you're right that this is more like what he's proposing. My main claim is that things will not move in that direction, rather the direction of the Mind's Eye paper I linked.

tablatom 4y ago
> learn everything the knowledge engine knows Isn't this an effectively infinite set? Wolfram Alpha could be said to know "all the numbers", and "all the formulas". > LLMs can learn structured domains too if they are well-represented in the training set But can they learn how to apply structured knowledge in precise ways? In mathematical or computational ways? I don't follow the field in great detail but the commentary I read seems to be saying this is not at all the case. And my own experiments with ChatGPT show it has no systematic grasp of logic.
- theptip 4y ago
  > Isn't this an effectively infinite set? No, the thing you'd want the LLM to be learning would be the rules. > But can they learn how to apply structured knowledge in precise ways? I personally believe: clearly yes, already. You can already get a LLM to generate code for simple logical problems. You can ask ChatGPT to modify a solution in a particular way, showing it has some understanding of the underlying logic, rather than just regurgitating solutions it saw. In other domains, you can already give IQ tests to GPT-N: https://lifearchitect.ai/ravens/ https://lifearchitect.ai/ravens/. Others have written in more detail than I could do justice to: https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis. I'd just note that a lot of commentators make quite simple errors of either goalpost-moving or a failure to extrapolate capabilities a year or two ahead. Of course, no linear or exponential growth curve continues indefinitely. But betting against this curve, now, seems to me a good way of losing money.
janalsncm 4y ago
> I am suggesting the LLM will (effectively) call out to a knowledge engine at training time, learn everything the knowledge engine knows, and render it obsolete. And when facts change, your only option will be to retrain. Since facts are always changing, you’ll always be training.
- theptip 4y ago
  I don't think most of the interesting knowledge encoded in Wolfram Alpha changes. Mathematics and pure Logic is true, and immutable. Most of Physics, ditto.
  - janalsncm 4y ago
    Thats true for the physics and math but some of the things are updated all the time. For example you can get a current weather report. And they have structured data about movies, TV shows, music, and notable people [1]. Every time any country has an election you’re going to retrain your model? That gets really expensive really fast. On top of that, the training process isn’t that trustworthy. There’s no guarantee your model won’t accidentally say that e.g. Obama is the president. All of this is to say that the best path forward is to translate questions into queries to an auditable knowledge base and then integrate the responses back into the conversation. It’s a couple years old but the best I’ve seen in this area is Retrieval Augmented Generation [2]. And even that’s imperfect, in my experience. [1] https://www.wolframalpha.com/ https://www.wolframalpha.com/ [2] https://arxiv.org/abs/2005.11401 https://arxiv.org/abs/2005.11401
    - theptip 4y ago
      Right, I see what you're getting at. I do agree that AI systems will need to be able to use oracles, "current weather" is a great example of something a human also looks up. The reason I want the model itself to learn Physics, Maths, etc. is that I think it is going to end up being critical to the challenge of actually developing logical reasoning on a par or above humans, and to gain a true "embodied understanding" of the real world. But yeah, it would be nice to have your architecture support updating facts without full retraining. One approach is to use an oracle as you note. Another would be to have systems do some sort of online learning (as humans do). (Why not both?) The advantage of the latter approach is that it allows an agent to deeply update their model of the world in response to new facts, as humans can sometimes do. Anything that I'm just pulling statelessly from an oracle cannot update the rest of my "mind". But this is perhaps a bit speculative; I agree in the short-term at least we'll see better performance with hybrid LLM + Oracle models. (As I noted, LaMDA already does this.) However I think that a big part of Wolfram's argument in the OP is that he thinks that an LLM can't learn Physics or Maths, or reliably learn static facts that a smart human might have memorized like distances between cities. And that's the position I was really trying to argue against. I think more scale and more data likely gets us way further than Wolfram wants to give credit for.
dsign 4y ago
>> I am suggesting the LLM will (effectively) call out to a knowledge engine at training time, learn everything the knowledge engine knows, and render it obsolete. You are suggesting that the LLM would "learn" to apply the rules built into the knowledge engine. I'm not as optimistic as to think that a statistical algorithm that uses random inputs would be reliable at applying deterministic rules. But for the sake of argument, let's assume that you are correct and that we can have an LLM replicate deterministic logic and exact computation, or that we can have it be right 95% of the time. That's basically the extent of human intellect[^1]: statistical processing for most common life situations, and deeper analytical thinking for some rather atypical cases (i.e., analyzing algorithm complexity, trying to demonstrate that the L4 Lagrange point is on the vertex of an equilateral triangle, or applying logic to forecast the stock market next week or how many degrees the global temperature will raise in the next two decades). Crucially, we are good at having computers check a proof, or simulate a system to ensure correctness, but before LLMs we were the only ones that could create that proof (by using socially learned heuristics to probe the proof space), design the system (by using knowledge passed down by fellow humans and assimilated by our "internal rule engines"), or come up with a definition of correctness (by doing a lot of inner and outer argumentation and possibly obtaining peer consensus). If we get an LLM do that as well, for most practical purposes we would have achieved AGI. If we are there already (or if we will be, in a few months or years), the world is going to look very different. Not necessarily in an apocalyptic way, but some priorities are going to shift[^2]. [^1]: Admittedly, there is also some biological characteristics that bias our intellectual processes in a certain way, but that's next-level madness and nobody is talking--for now--about giving those to an LLM. [^2]: If you could have at your beck and call a general intellect engine that could build entire programs and systems for you, what would you have it build?

tablatom 4y ago

> learn everything the knowledge engine knows Isn't this an effectively infinite set? Wolfram Alpha could be said to know "all the numbers", and "all the formulas". > LLMs can learn structured domains too if they are well-represented in the training set But can they learn how to apply structured knowledge in precise ways? In mathematical or computational ways? I don't follow the field in great detail but the commentary I read seems to be saying this is not at all the case. And my own experiments with ChatGPT show it has no systematic grasp of logic.

theptip 4y ago
> Isn't this an effectively infinite set? No, the thing you'd want the LLM to be learning would be the rules. > But can they learn how to apply structured knowledge in precise ways? I personally believe: clearly yes, already. You can already get a LLM to generate code for simple logical problems. You can ask ChatGPT to modify a solution in a particular way, showing it has some understanding of the underlying logic, rather than just regurgitating solutions it saw. In other domains, you can already give IQ tests to GPT-N: https://lifearchitect.ai/ravens/ https://lifearchitect.ai/ravens/. Others have written in more detail than I could do justice to: https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis. I'd just note that a lot of commentators make quite simple errors of either goalpost-moving or a failure to extrapolate capabilities a year or two ahead. Of course, no linear or exponential growth curve continues indefinitely. But betting against this curve, now, seems to me a good way of losing money.

janalsncm 4y ago

> I am suggesting the LLM will (effectively) call out to a knowledge engine at training time, learn everything the knowledge engine knows, and render it obsolete. And when facts change, your only option will be to retrain. Since facts are always changing, you’ll always be training.

theptip 4y ago
I don't think most of the interesting knowledge encoded in Wolfram Alpha changes. Mathematics and pure Logic is true, and immutable. Most of Physics, ditto.
- janalsncm 4y ago
  Thats true for the physics and math but some of the things are updated all the time. For example you can get a current weather report. And they have structured data about movies, TV shows, music, and notable people [1]. Every time any country has an election you’re going to retrain your model? That gets really expensive really fast. On top of that, the training process isn’t that trustworthy. There’s no guarantee your model won’t accidentally say that e.g. Obama is the president. All of this is to say that the best path forward is to translate questions into queries to an auditable knowledge base and then integrate the responses back into the conversation. It’s a couple years old but the best I’ve seen in this area is Retrieval Augmented Generation [2]. And even that’s imperfect, in my experience. [1] https://www.wolframalpha.com/ https://www.wolframalpha.com/ [2] https://arxiv.org/abs/2005.11401 https://arxiv.org/abs/2005.11401
  - theptip 4y ago
    Right, I see what you're getting at. I do agree that AI systems will need to be able to use oracles, "current weather" is a great example of something a human also looks up. The reason I want the model itself to learn Physics, Maths, etc. is that I think it is going to end up being critical to the challenge of actually developing logical reasoning on a par or above humans, and to gain a true "embodied understanding" of the real world. But yeah, it would be nice to have your architecture support updating facts without full retraining. One approach is to use an oracle as you note. Another would be to have systems do some sort of online learning (as humans do). (Why not both?) The advantage of the latter approach is that it allows an agent to deeply update their model of the world in response to new facts, as humans can sometimes do. Anything that I'm just pulling statelessly from an oracle cannot update the rest of my "mind". But this is perhaps a bit speculative; I agree in the short-term at least we'll see better performance with hybrid LLM + Oracle models. (As I noted, LaMDA already does this.) However I think that a big part of Wolfram's argument in the OP is that he thinks that an LLM can't learn Physics or Maths, or reliably learn static facts that a smart human might have memorized like distances between cities. And that's the position I was really trying to argue against. I think more scale and more data likely gets us way further than Wolfram wants to give credit for.

dsign 4y ago

>> I am suggesting the LLM will (effectively) call out to a knowledge engine at training time, learn everything the knowledge engine knows, and render it obsolete. You are suggesting that the LLM would "learn" to apply the rules built into the knowledge engine. I'm not as optimistic as to think that a statistical algorithm that uses random inputs would be reliable at applying deterministic rules. But for the sake of argument, let's assume that you are correct and that we can have an LLM replicate deterministic logic and exact computation, or that we can have it be right 95% of the time. That's basically the extent of human intellect[^1]: statistical processing for most common life situations, and deeper analytical thinking for some rather atypical cases (i.e., analyzing algorithm complexity, trying to demonstrate that the L4 Lagrange point is on the vertex of an equilateral triangle, or applying logic to forecast the stock market next week or how many degrees the global temperature will raise in the next two decades). Crucially, we are good at having computers check a proof, or simulate a system to ensure correctness, but before LLMs we were the only ones that could create that proof (by using socially learned heuristics to probe the proof space), design the system (by using knowledge passed down by fellow humans and assimilated by our "internal rule engines"), or come up with a definition of correctness (by doing a lot of inner and outer argumentation and possibly obtaining peer consensus). If we get an LLM do that as well, for most practical purposes we would have achieved AGI. If we are there already (or if we will be, in a few months or years), the world is going to look very different. Not necessarily in an apocalyptic way, but some priorities are going to shift[^2]. [^1]: Admittedly, there is also some biological characteristics that bias our intellectual processes in a certain way, but that's next-level madness and nobody is talking--for now--about giving those to an LLM. [^2]: If you could have at your beck and call a general intellect engine that could build entire programs and systems for you, what would you have it build?