2 ms·

There's a distinction between "a model" and the chain of tools and models you employ when asking something on chatgpt.com or any of the consumer facing alternat

by berkes 2y ago

There's a distinction between "a model" and the chain of tools and models you employ when asking something on chatgpt.com or any of the consumer facing alternatives. The latter is a chain of models, some specialized in question dissecting, some specialized in choosing the right models and tools (i.e: there's a calculation in there, lets push that part to a simple python function that can actually count stuff, and pull the rest through a generic LLM). I experiment with such toolchains myself and it's baffling how fast the complexity of all this is becoming. A very simple example would be "question" -> "does_it_want_code_generated.model" -[yes]-> specialized_code_generator.model | -[no]-> specialized_english_generator.model" So, sure: a model has no "knowledge", and nor does a chain of tools. But having e.g. a model specialized (ie. trained on or enriched with) all scientific papers ever, or maybe even a vector DB with all that data, somewhere in the toolchain that is in charge of either finding the "very likely references" or denying an answer would help a lot. It would for me.

Yizahi 2y ago
Sure, chains of networks can guess at the "passable" answer much better/faster/cheaper etc. But that doesn't remove the core issue, that none of the sub-networks or decision trees can understand what it generates, and so it can't abort its work and output "no answer" or something similar. The whole premise of original request was that user raises a task for NN which has a verifiable (maybe partially) answer. He sees incorrect answer and wishes that a "failure" was displayed instead. But NN can't verify correctness of it's output. After all G in GPT stands for Generative.
- berkes 2y ago
  My simple RAG setup has a steps that will return "We don't have this information" if e.g. our vector DB returns entries with far too low relevancy scores or if the response from the LLM fails to add certain attributes in its answer and so on. Edit: TBC: these "steps" aren't LLMS or other models. They're simple code with simple if/elses and an accidental regex. Again: an LLM/NN indeed has no "understanding" of what it creates. Especially the LLMs that are "just" statistical models. But the tooling around it, the entire chain can very well handle this.
  - robwwilliams 2y ago
    Perfect. We need that society of modules.

by berkes 2y ago

Yizahi 2y ago

Sure, chains of networks can guess at the "passable" answer much better/faster/cheaper etc. But that doesn't remove the core issue, that none of the sub-networks or decision trees can understand what it generates, and so it can't abort its work and output "no answer" or something similar. The whole premise of original request was that user raises a task for NN which has a verifiable (maybe partially) answer. He sees incorrect answer and wishes that a "failure" was displayed instead. But NN can't verify correctness of it's output. After all G in GPT stands for Generative.

berkes 2y ago
My simple RAG setup has a steps that will return "We don't have this information" if e.g. our vector DB returns entries with far too low relevancy scores or if the response from the LLM fails to add certain attributes in its answer and so on. Edit: TBC: these "steps" aren't LLMS or other models. They're simple code with simple if/elses and an accidental regex. Again: an LLM/NN indeed has no "understanding" of what it creates. Especially the LLMs that are "just" statistical models. But the tooling around it, the entire chain can very well handle this.
- robwwilliams 2y ago
  Perfect. We need that society of modules.