4 ms·

Ive used ChatGPT to make short factual videos for YouTube and honestly it's a bit worrying with supposed 'facts' I would not suggest anyone to use ChatGPT outp

by herbst 4y ago

Ive used ChatGPT to make short factual videos for YouTube and honestly it's a bit worrying with supposed 'facts' I would not suggest anyone to use ChatGPT outputs for actual knowledge at this point.

magicalhippo 4y ago
I've not used ChatGPT yet (almost feel like a luddite), what happens when you tell it that it's wrong? Will it get it right after a retry or two?
- mepian 4y ago
  Sometimes by accident it will, but most of the time it keeps making things up with no regard to factual and logical correctness. It has no notion of truth.
aflag 4y ago
You can use it for that, but you have to test its claims. It's specially helpful with questions that would take a lot of research to find the answer, but which are easier to test. That could be a good thing overall. If people get used to always testing everything anyone tells them, people will be a lot better informed.
- visarga 4y ago
  I think LLM checking is going to be the most important research direction this year. Generative models are worthless without verification. It relates to deciding truth and dealing with fake information, synthetic text and spam. Google hasn't been able to solve it in the last decade, but some people have an idea: > "Discovering Latent Knowledge in Language Models Without Supervision" Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect. We propose circumventing this issue by directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. https://arxiv.org/abs/2212.03827 https://arxiv.org/abs/2212.03827 In other words the model already tries to predict the truth because it is useful in next token prediction, but we need to find a way to detect the 'truth alignment' in its activations.
Baeocystin 4y ago
I think getting in to the habit of fact-checking everything LLMs come up with when used to generate is probably an excellent idea. That being said, I haven't really seen any confabulation when using it to summarize a supplied text.
julius 4y ago
I asked it how to do something with an AWS cli tool. ChatGPT invented a new parameter that looked like a programmer would come up with it and it would do exactly what I was looking for (I assumed many people had my problem before). aws cloudfront update-distribution --id <distribution-id> --distribution-config <new-config> --no-reset-origin-access-identity Took me a while to figure out that the parameter --no-reset-origin-access-identity was not only not working. But it did never exist on any version of the cli tool.
- te_chris 4y ago
  Did you not think to check the docs? (Srs question)
  - coeneedell 4y ago
    At least in my experience, the aws docs are difficult to understand at best, and horrifically outdated and contradictory at worst.
  - julius 4y ago
    ChatGPT was my first choice just because I want to know if it is any good at helping me with my work. But for this problem (CF-Distribution lost OAC settings when updating the root file), all the google fu in the world did not help me. It turned out that I had to update my aws-cli and my problem went away. Apparently no one else on the internet had that problem, so only my gut could help me figure it out.
- mtmail 4y ago
  Same here. ChatGPT invented a feature of our API and wrote an example script. User then complained to us that it's not working.
- belter 4y ago
  I asked it to create a COBOL program to connect to AWS and create an S3 Bucket... https://news.ycombinator.com/item?id=33991767 https://news.ycombinator.com/item?id=33991767
williamcotton 4y ago
ChatGPT works best with analytic prompts that respond with translations and not synthesis. Meaning, if all the facts exist in the prompt then the likelihood of synthesizing fiction is diminished. There are a number of ways to use the principle of analytic augmentation to add most or if not all of the facts required for a truthful response, ranging from simple “prompt engineering” to evaluating code to document embedding in latent space. For example, if you use prompt engineering to k-shot a task to turn math word problems into executable JavaScript, meaning LLMs are only translators and the computations are done by a software interpreter, then the results are much more likely to be truthful. Sampling from a number of variations on a prompt can lead to a more accurate outcome if say 1:10 times the translation attempt has a different answer.

by herbst 4y ago

Ive used ChatGPT to make short factual videos for YouTube and honestly it's a bit worrying with supposed 'facts' I would not suggest anyone to use ChatGPT outputs for actual knowledge at this point.

magicalhippo 4y ago

I've not used ChatGPT yet (almost feel like a luddite), what happens when you tell it that it's wrong? Will it get it right after a retry or two?

mepian 4y ago
Sometimes by accident it will, but most of the time it keeps making things up with no regard to factual and logical correctness. It has no notion of truth.

aflag 4y ago

You can use it for that, but you have to test its claims. It's specially helpful with questions that would take a lot of research to find the answer, but which are easier to test. That could be a good thing overall. If people get used to always testing everything anyone tells them, people will be a lot better informed.

visarga 4y ago
I think LLM checking is going to be the most important research direction this year. Generative models are worthless without verification. It relates to deciding truth and dealing with fake information, synthetic text and spam. Google hasn't been able to solve it in the last decade, but some people have an idea: > "Discovering Latent Knowledge in Language Models Without Supervision" Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect. We propose circumventing this issue by directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. https://arxiv.org/abs/2212.03827 https://arxiv.org/abs/2212.03827 In other words the model already tries to predict the truth because it is useful in next token prediction, but we need to find a way to detect the 'truth alignment' in its activations.

Baeocystin 4y ago

I think getting in to the habit of fact-checking everything LLMs come up with when used to generate is probably an excellent idea. That being said, I haven't really seen any confabulation when using it to summarize a supplied text.

julius 4y ago

I asked it how to do something with an AWS cli tool. ChatGPT invented a new parameter that looked like a programmer would come up with it and it would do exactly what I was looking for (I assumed many people had my problem before). aws cloudfront update-distribution --id <distribution-id> --distribution-config <new-config> --no-reset-origin-access-identity Took me a while to figure out that the parameter --no-reset-origin-access-identity was not only not working. But it did never exist on any version of the cli tool.

te_chris 4y ago
Did you not think to check the docs? (Srs question)
- coeneedell 4y ago
  At least in my experience, the aws docs are difficult to understand at best, and horrifically outdated and contradictory at worst.
- julius 4y ago
  ChatGPT was my first choice just because I want to know if it is any good at helping me with my work. But for this problem (CF-Distribution lost OAC settings when updating the root file), all the google fu in the world did not help me. It turned out that I had to update my aws-cli and my problem went away. Apparently no one else on the internet had that problem, so only my gut could help me figure it out.
mtmail 4y ago
Same here. ChatGPT invented a feature of our API and wrote an example script. User then complained to us that it's not working.
belter 4y ago
I asked it to create a COBOL program to connect to AWS and create an S3 Bucket... https://news.ycombinator.com/item?id=33991767 https://news.ycombinator.com/item?id=33991767

williamcotton 4y ago

ChatGPT works best with analytic prompts that respond with translations and not synthesis. Meaning, if all the facts exist in the prompt then the likelihood of synthesizing fiction is diminished. There are a number of ways to use the principle of analytic augmentation to add most or if not all of the facts required for a truthful response, ranging from simple “prompt engineering” to evaluating code to document embedding in latent space. For example, if you use prompt engineering to k-shot a task to turn math word problems into executable JavaScript, meaning LLMs are only translators and the computations are done by a software interpreter, then the results are much more likely to be truthful. Sampling from a number of variations on a prompt can lead to a more accurate outcome if say 1:10 times the translation attempt has a different answer.