32 ms·
Gpt-4 can do formal logic and symbolic manipulation. I really don’t understand your comment.
by computerex 4y ago
Gpt-4 can do formal logic and symbolic manipulation. I really don’t understand your comment.
- simonw 4y agoI was responding to "LLMs need a set of axioms and formal logic to establish truthfulness of arbitrary statements." Anyone who figures out a way to "establish truthfulness of arbitrary statements" in large language models will be making a HUGE breakthrough.
- computerex 4y agoLanguage models can establish truthfulness in a formal logic system. There is a lot of literature that indicates this. Look at this for instance: https://openreview.net/forum?id=qFVVBzXxR2V https://openreview.net/forum?id=qFVVBzXxR2V And you can test this quite easily with a prompt like this. gpt-4 answers this correctly without any tuning: you have 2 boxes, box a and box b. Box a has 3 index cards, each with a giant letter printed in the middle with a small caption below the letter. Based on the contents of Box A and Box B, infer what the blank index card in Box B should most likely say The contents of box a: letter P, caption P:O letter T, caption T:S letter J, caption J:K The contents of box b: index card with O printed in the middle index card with S printed in the middle blank index card
- simonw 4y agoI interpreted "establish truthfulness of arbitrary statements" as referring to the challenge of telling things like if a statement like "mars has a population of 12 million" is truthful or not, given the available training data for the LLM.
- computerex 4y agoThat's also "solved". Yes models exhibit hallucinations, but chain of thought prompting and ReAct prompting improve accuracy on factual tasks greatly, particuarly in a data augmented generation setting. We are finding new and better ways to prompt language models. Scaling up the models even further will also improve performance.