5 ms·

The so called AI can't "know". It doesn't have understanding if the generated text is an answer or of it isn't. You can't force that instruction on a neural net

by Yizahi 2y ago

The so called AI can't "know". It doesn't have understanding if the generated text is an answer or of it isn't. You can't force that instruction on a neural network, at best it just adjusts generated text slightly and you think that it somehow started understanding.

berkes 2y ago
There's a distinction between "a model" and the chain of tools and models you employ when asking something on chatgpt.com or any of the consumer facing alternatives. The latter is a chain of models, some specialized in question dissecting, some specialized in choosing the right models and tools (i.e: there's a calculation in there, lets push that part to a simple python function that can actually count stuff, and pull the rest through a generic LLM). I experiment with such toolchains myself and it's baffling how fast the complexity of all this is becoming. A very simple example would be "question" -> "does_it_want_code_generated.model" -[yes]-> specialized_code_generator.model | -[no]-> specialized_english_generator.model" So, sure: a model has no "knowledge", and nor does a chain of tools. But having e.g. a model specialized (ie. trained on or enriched with) all scientific papers ever, or maybe even a vector DB with all that data, somewhere in the toolchain that is in charge of either finding the "very likely references" or denying an answer would help a lot. It would for me.
- Yizahi 2y ago
  Sure, chains of networks can guess at the "passable" answer much better/faster/cheaper etc. But that doesn't remove the core issue, that none of the sub-networks or decision trees can understand what it generates, and so it can't abort its work and output "no answer" or something similar. The whole premise of original request was that user raises a task for NN which has a verifiable (maybe partially) answer. He sees incorrect answer and wishes that a "failure" was displayed instead. But NN can't verify correctness of it's output. After all G in GPT stands for Generative.
  - berkes 2y ago
    My simple RAG setup has a steps that will return "We don't have this information" if e.g. our vector DB returns entries with far too low relevancy scores or if the response from the LLM fails to add certain attributes in its answer and so on. Edit: TBC: these "steps" aren't LLMS or other models. They're simple code with simple if/elses and an accidental regex. Again: an LLM/NN indeed has no "understanding" of what it creates. Especially the LLMs that are "just" statistical models. But the tooling around it, the entire chain can very well handle this.
    - robwwilliams 2y ago
      Perfect. We need that society of modules.
baq 2y ago
How confident can you be in this? Have you analyzed what exactly the billions of weights do? I’ve got my opinions about what LLMs are and what they aren’t, but I don’t confidently claim that they must be such. There’s a lot of stuff in those weights.
- Q6T46nT668w6i3m 2y ago
  I’m confident that there’s no magic and I’ve spent years understanding “what the weights do.” You’re describing weights as magic and they are not.
  - Nevermark 2y ago
    Except the weights form complex relationships in order to reproduce very human usable responses. You can't look at weights and say it is doing this, or not doing that, unless you dive into a particular model. Especially when you have billions of weights. These models are finding general patterns that apply across all kinds of subjects. Patterns they aptly recognize and weave in all kinds of combinations. They are sensibly conversing on virtually every topic known to human kind. And can talk sensibly about any two topics, in conjunction. There is magic. Not mystic magic, but we are going to learn a lot as we decode how their style of processing (after training) works. We don't have a good theory of how either LLM's or we "reason" in the intuitive sense. And yet they learn to do it. It will inspire improved and more efficient architectures. I have also spent many years looking at weights!
    - robwwilliams 2y ago
      Love your end. I have have spent four decades looking at real neurons, real synapses, and real axons and I can tell you with complete confidence that we are all just zombies.
      - Nevermark 2y ago
        Spectators riding around in zombies! Imagining we are really doing everything it does automatically including learning via algorithms we have only vague understandings of. That is a strange thought. I could look at all my own brain's neurons, even with a heads up display showing all the activity clearly, and have no idea that it was me.
hatthew 2y ago
Can you clarify what definition of "understanding" you're using here?

by Yizahi 2y ago

berkes 2y ago

There's a distinction between "a model" and the chain of tools and models you employ when asking something on chatgpt.com or any of the consumer facing alternatives. The latter is a chain of models, some specialized in question dissecting, some specialized in choosing the right models and tools (i.e: there's a calculation in there, lets push that part to a simple python function that can actually count stuff, and pull the rest through a generic LLM). I experiment with such toolchains myself and it's baffling how fast the complexity of all this is becoming. A very simple example would be "question" -> "does_it_want_code_generated.model" -[yes]-> specialized_code_generator.model | -[no]-> specialized_english_generator.model" So, sure: a model has no "knowledge", and nor does a chain of tools. But having e.g. a model specialized (ie. trained on or enriched with) all scientific papers ever, or maybe even a vector DB with all that data, somewhere in the toolchain that is in charge of either finding the "very likely references" or denying an answer would help a lot. It would for me.

Yizahi 2y ago
Sure, chains of networks can guess at the "passable" answer much better/faster/cheaper etc. But that doesn't remove the core issue, that none of the sub-networks or decision trees can understand what it generates, and so it can't abort its work and output "no answer" or something similar. The whole premise of original request was that user raises a task for NN which has a verifiable (maybe partially) answer. He sees incorrect answer and wishes that a "failure" was displayed instead. But NN can't verify correctness of it's output. After all G in GPT stands for Generative.
- berkes 2y ago
  My simple RAG setup has a steps that will return "We don't have this information" if e.g. our vector DB returns entries with far too low relevancy scores or if the response from the LLM fails to add certain attributes in its answer and so on. Edit: TBC: these "steps" aren't LLMS or other models. They're simple code with simple if/elses and an accidental regex. Again: an LLM/NN indeed has no "understanding" of what it creates. Especially the LLMs that are "just" statistical models. But the tooling around it, the entire chain can very well handle this.
  - robwwilliams 2y ago
    Perfect. We need that society of modules.

baq 2y ago

How confident can you be in this? Have you analyzed what exactly the billions of weights do? I’ve got my opinions about what LLMs are and what they aren’t, but I don’t confidently claim that they must be such. There’s a lot of stuff in those weights.

Q6T46nT668w6i3m 2y ago
I’m confident that there’s no magic and I’ve spent years understanding “what the weights do.” You’re describing weights as magic and they are not.
- Nevermark 2y ago
  Except the weights form complex relationships in order to reproduce very human usable responses. You can't look at weights and say it is doing this, or not doing that, unless you dive into a particular model. Especially when you have billions of weights. These models are finding general patterns that apply across all kinds of subjects. Patterns they aptly recognize and weave in all kinds of combinations. They are sensibly conversing on virtually every topic known to human kind. And can talk sensibly about any two topics, in conjunction. There is magic. Not mystic magic, but we are going to learn a lot as we decode how their style of processing (after training) works. We don't have a good theory of how either LLM's or we "reason" in the intuitive sense. And yet they learn to do it. It will inspire improved and more efficient architectures. I have also spent many years looking at weights!
  - robwwilliams 2y ago
    Love your end. I have have spent four decades looking at real neurons, real synapses, and real axons and I can tell you with complete confidence that we are all just zombies.
    - Nevermark 2y ago
      Spectators riding around in zombies! Imagining we are really doing everything it does automatically including learning via algorithms we have only vague understandings of. That is a strange thought. I could look at all my own brain's neurons, even with a heads up display showing all the activity clearly, and have no idea that it was me.

hatthew 2y ago

Can you clarify what definition of "understanding" you're using here?