4 ms·

Large language models are one particular implementation of machine intelligence optimized for generating language but not necessarily for being true or for bein

Large language models are one particular implementation of machine intelligence optimized for generating language but not necessarily for being true or for being grounded in sourceable to reference material or facts. I think we're taking models that have been optimized for a particular thing and repurposing them towards new tasks and then assuming that the principles of large language models as we have developed them are just universally inherent properties of AI, which I think is not at all the case. Conceivably, you absolutely could train models where their own built-in metric of success involves a notion of appropriateness of citations. The models we have simply don't do that, but I don't think we should proceed to the assumption that we are witnessing the future of AI with hallucinations everywhere, I just think it's a lazy repurposing of systems that were not initially designed with these things in mind.

Terr_ 2y ago
> optimized for a particular thing and repurposing them A lot of human exuberance (and thus investment) is riding on the questionable idea that a really good text fragments correlation specialist system can usefully impersonate a generalist AI. ("LLM, which rocks are the best to eat?") Even worse: Some people assume an exceptional text-specialist model will usefully meta-impersonate a generalist model impersonating a specific and different kind of specialist! ("LLM, do my taxes.")
thfuran 2y ago
How many of the large systems in the world (civil infrastructure, bureaucracies, the internet, etc) don't involve a great deal of repurposing previously present systems for purposes beyond their original design intent? I don't think we're guaranteed to be stuck with hallucinating AI embedded in everything, but if something gets popular enough to work its way into a lot of tech stacks, it can easily persist well beyond the point where anyone designing things from the ground up would include it simply because it was already there.
duxup 2y ago
Yeah as far as my personal use cases, code or basic Q & A that sort of chains together naturally. It's easy enough for me to quick test these ideas or code. I can typically eyeball it when it is wrong and if not then, I'll run it and see. Done. It works for me, but mostly because I can easily validate and I'm confidant in quick / painless validations on my end. Something in depth that involves a great deal more multi step reasoning to get from point A to B, I've got my doubts. If anything that's where I see AI wander off the path into a mess.
fallingknife 2y ago
But how do you do that? The model is a pattern recognition engine and the problem we call "hallucination" is actually that the model is recognizing patterns from data that is similar to, but not an exact match to what we want. In order to eliminate this issue with current architecture it would require a model trained on only the specific data set we want answers to be drawn from. But that isn't a viable solution because just to get responses that aren't total nonsense requires a training set so large as to make this impossible for types of specialized data.
- jimbokun 2y ago
  Maybe two models? One like current LLMs, generating the usual bullshit. A second model trained to map output from the first model to reliable citations or mapping to some value from 0 to 1 predicting the confidence of the models accuracy. Clearly I am just bullshitting myself here, I don't know how to train the second model. Something mapping text to reliable sources...(waves hands)
  - Jensson 2y ago
    > Something mapping text to reliable sources...(waves hands) You mean basically Google search? What you want is an intelligent search engine, no such search engine exist today but not due to lack of trying, this is a trillion dollar problem.
    - thereisnospork 2y ago
      Not to say that it is easy in absolute terms, but I'd argue that true/false'ing a statement, e.g. "humans should eat 1 rock a day" is a categorically easier problem than answering "What should humans eat"? For fun/example I asked gpt3.5 "What percent of dieticians would suggest eating one rock a day is good for your health?" And got a pretty solid if wordy 'none'.
  - fallingknife 2y ago
    But how do you do "reliable citations" with the current architecture? You still have the problem that it is at its core a pattern recognition engine. It will just be "looks similar to all the reliable citations in the training set for similar subjects" not "this is the correct citation for your specific query."
morsecodist 2y ago
Designing the metric of success is the hard part. The magic of LLMs is that they can learn a lot with unlabeled data and a relatively simple cost function. They can be trained the mountains of data we just have lying around without a ton of additional labeling or making clever domain-specific metrics. If this approach doesn't really generalize well, it negates a lot of the usefulness of these methods and suggests we are further away than we might think.
- scotty79 2y ago
  I think way out of this is training models on multimodal data where one of the reasoning modes are reasoning graphs or knowledge graphs or schematics. We might gradually improve constructing graphs from data like texts, images and speech. And we could create systems that correctly create synthetic data from generated graphs to serve as a training data for large multimodal model.

Terr_ 2y ago

> optimized for a particular thing and repurposing them A lot of human exuberance (and thus investment) is riding on the questionable idea that a really good text fragments correlation specialist system can usefully impersonate a generalist AI. ("LLM, which rocks are the best to eat?") Even worse: Some people assume an exceptional text-specialist model will usefully meta-impersonate a generalist model impersonating a specific and different kind of specialist! ("LLM, do my taxes.")

thfuran 2y ago

How many of the large systems in the world (civil infrastructure, bureaucracies, the internet, etc) don't involve a great deal of repurposing previously present systems for purposes beyond their original design intent? I don't think we're guaranteed to be stuck with hallucinating AI embedded in everything, but if something gets popular enough to work its way into a lot of tech stacks, it can easily persist well beyond the point where anyone designing things from the ground up would include it simply because it was already there.

duxup 2y ago

Yeah as far as my personal use cases, code or basic Q & A that sort of chains together naturally. It's easy enough for me to quick test these ideas or code. I can typically eyeball it when it is wrong and if not then, I'll run it and see. Done. It works for me, but mostly because I can easily validate and I'm confidant in quick / painless validations on my end. Something in depth that involves a great deal more multi step reasoning to get from point A to B, I've got my doubts. If anything that's where I see AI wander off the path into a mess.

fallingknife 2y ago

But how do you do that? The model is a pattern recognition engine and the problem we call "hallucination" is actually that the model is recognizing patterns from data that is similar to, but not an exact match to what we want. In order to eliminate this issue with current architecture it would require a model trained on only the specific data set we want answers to be drawn from. But that isn't a viable solution because just to get responses that aren't total nonsense requires a training set so large as to make this impossible for types of specialized data.

jimbokun 2y ago
Maybe two models? One like current LLMs, generating the usual bullshit. A second model trained to map output from the first model to reliable citations or mapping to some value from 0 to 1 predicting the confidence of the models accuracy. Clearly I am just bullshitting myself here, I don't know how to train the second model. Something mapping text to reliable sources...(waves hands)
- Jensson 2y ago
  > Something mapping text to reliable sources...(waves hands) You mean basically Google search? What you want is an intelligent search engine, no such search engine exist today but not due to lack of trying, this is a trillion dollar problem.
  - thereisnospork 2y ago
    Not to say that it is easy in absolute terms, but I'd argue that true/false'ing a statement, e.g. "humans should eat 1 rock a day" is a categorically easier problem than answering "What should humans eat"? For fun/example I asked gpt3.5 "What percent of dieticians would suggest eating one rock a day is good for your health?" And got a pretty solid if wordy 'none'.
- fallingknife 2y ago
  But how do you do "reliable citations" with the current architecture? You still have the problem that it is at its core a pattern recognition engine. It will just be "looks similar to all the reliable citations in the training set for similar subjects" not "this is the correct citation for your specific query."

morsecodist 2y ago

Designing the metric of success is the hard part. The magic of LLMs is that they can learn a lot with unlabeled data and a relatively simple cost function. They can be trained the mountains of data we just have lying around without a ton of additional labeling or making clever domain-specific metrics. If this approach doesn't really generalize well, it negates a lot of the usefulness of these methods and suggests we are further away than we might think.

scotty79 2y ago
I think way out of this is training models on multimodal data where one of the reasoning modes are reasoning graphs or knowledge graphs or schematics. We might gradually improve constructing graphs from data like texts, images and speech. And we could create systems that correctly create synthetic data from generated graphs to serve as a training data for large multimodal model.