2 ms·
I'm not sure if this would work, as you cannot always define the factual accuracy on a token-by-token basis. However I wonder if having the LLM answer question
by xg15 4y ago
I'm not sure if this would work, as you cannot always define the factual accuracy on a token-by-token basis.
However I wonder if having the LLM answer questions based on the knowledge baked in the model is the way forward at all: You quickly run into issues with up-to-date information, as you'd have to do a possibly expensive retraining every few weeks at least - and you also have no way of controlling which information exactly is stored exactly and how the models combines it.
I think an interesting alternative approach could be to split the bot up into two "agents" and one backend database:
- One model which receives user input and translates it into a machine-readable well-defined representation e.g. a series of SQL or SPARQL queries, a series of API calls or whatever.
- Then have a wholly non-AI backend that validates and executes those queries and delivers a result, in machine-readable format as well.
- Finally have a second model (or the first model with different finetuning/prompt) which can translate the query result back into a freetext sentence, which is then delivered to the user.
This way, you can keep tighter control about what exactly the model is answering and you can also update the underlying knowledgebase independently of the model.
- CuriouslyC 4y agoIt wouldn't be on a token by token basis, but rather "common" token sequence tuples. If you generate a histogram of common 5-8 grams, and filter out all the ones that are just linguistic filler and keep the ones that are "factual," then feed those "factual" n-grams to reviewers (via captcha/turk/game) to get a distribution of accuracy scores that you then condition the model on, so it can produce "accuracy" estimates for token sequences, and further condition on high accuracy scores for generation. for example (butchered for clarity), (["The", "2022", "election", "was", "stolen"], accuracy=normal(0.0001, 0.1)).
- ianmcgowan 4y agoThat sounds like the blog post by Steven Wolfram recently [1] where he proposes having two models cooperating - one that "knows" facts (has computational knowledge in that post), and the other that controls the language interpretation and decides which facts are relevant. It's interesting to go further and speculate that our "mind" is made up of cooperating sub-systems like this. https://writings.stephenwolfram.com/2023/01/wolframalpha-as-the-way-to-bring-computational-knowledge-superpowers-to-chatgpt/ https://writings.stephenwolfram.com/2023/01/wolframalpha-as-...