5 ms·

I see trust as the big issue. Yes, it gives very reasonable answers. But how do I know it is not BSing, defrauding, or tricking me? You can ask for it to reveal

by mertd 4y ago

I see trust as the big issue. Yes, it gives very reasonable answers. But how do I know it is not BSing, defrauding, or tricking me? You can ask for it to reveal its sources, but then you're back to Google style knowledge gathering of checking websites and judging credibility.

macintux 4y ago
There was a Twitter thread discussed here just a day or two ago where it was giving serious, in-depth, plausible answers to questions around Typescript. Yet apparently those answers were quite wrong. https://news.ycombinator.com/item?id=33817682 https://news.ycombinator.com/item?id=33817682 That trust factor is huge. I had this argument in the thread: when a human isn't certain of the right answer, they will (typically) provide answers, an explanation for their reasoning, and cautions about where they could be wrong. An AI that sounds confident 100% of the time but is occasionally very wrong is going to be a real problem.
- underwater 4y ago
  Why can't you hook it up to canonical references? If you asked it how to search those references then I feel certain it would be able to navigate to, parse and summarise the answers with ease.
carl_dr 4y ago
> How many ways can you permute the letters in the word “safer”? < There are 24 possible ways to permute the letters in the word "safer". This can be calculated using the formula for calculating the number of permutations of a set of items, which is n! (factorial n), where n is the number of items in the set. In this case, there are 5 letters in the word "safer", so the number of permutations would be 5! = 5 x 4 x 3 x 2 x 1 = 120. However, since the letters "a", "e", and "r" are repeated in the word, we need to divide the total number of permutations by the number of permutations of each of these repeated letters. In this case, there are 3 "a"s, 2 "e"s, and 2 "r"s, so the number of permutations would be 120 / (3! x 2! x 2!) = 24 Starts out just wrong, then explains the working and correct answer, but just completely loses it by the end. That calculation!
- coffeebeqn 4y ago
  It seems to not calculate math formulas. Probably they turned that off because people would throw absurdly expensive operations at it
visarga 4y ago
That's exactly how to tackle the problem. Using a search engine to ground it in real data. It's called a "retrieval augmented language model" and it basically means it first does a search and selects a few relevant passages, then solves the task using that additional information. The problem of verifying truthfulness is separate, but retrieval gives you grounding and references, both being necessary in order to use the model as a research assistant or teacher. It's not a fundamental limitation, it will be standard issue soon enough - web enabled AI bot. It even has a flag called "Browsing: disabled" in its prompt, suggesting it will be enabled soon, probably after they find it safe enough.
pcthrowaway 4y ago
How is this any better than what you'll get from Google? Now that the cat's out of the bag, a lot of results may even be generated

by mertd 4y ago

macintux 4y ago

There was a Twitter thread discussed here just a day or two ago where it was giving serious, in-depth, plausible answers to questions around Typescript. Yet apparently those answers were quite wrong. https://news.ycombinator.com/item?id=33817682 https://news.ycombinator.com/item?id=33817682 That trust factor is huge. I had this argument in the thread: when a human isn't certain of the right answer, they will (typically) provide answers, an explanation for their reasoning, and cautions about where they could be wrong. An AI that sounds confident 100% of the time but is occasionally very wrong is going to be a real problem.

underwater 4y ago
Why can't you hook it up to canonical references? If you asked it how to search those references then I feel certain it would be able to navigate to, parse and summarise the answers with ease.

carl_dr 4y ago

> How many ways can you permute the letters in the word “safer”? < There are 24 possible ways to permute the letters in the word "safer". This can be calculated using the formula for calculating the number of permutations of a set of items, which is n! (factorial n), where n is the number of items in the set. In this case, there are 5 letters in the word "safer", so the number of permutations would be 5! = 5 x 4 x 3 x 2 x 1 = 120. However, since the letters "a", "e", and "r" are repeated in the word, we need to divide the total number of permutations by the number of permutations of each of these repeated letters. In this case, there are 3 "a"s, 2 "e"s, and 2 "r"s, so the number of permutations would be 120 / (3! x 2! x 2!) = 24 Starts out just wrong, then explains the working and correct answer, but just completely loses it by the end. That calculation!

coffeebeqn 4y ago
It seems to not calculate math formulas. Probably they turned that off because people would throw absurdly expensive operations at it

visarga 4y ago

That's exactly how to tackle the problem. Using a search engine to ground it in real data. It's called a "retrieval augmented language model" and it basically means it first does a search and selects a few relevant passages, then solves the task using that additional information. The problem of verifying truthfulness is separate, but retrieval gives you grounding and references, both being necessary in order to use the model as a research assistant or teacher. It's not a fundamental limitation, it will be standard issue soon enough - web enabled AI bot. It even has a flag called "Browsing: disabled" in its prompt, suggesting it will be enabled soon, probably after they find it safe enough.

pcthrowaway 4y ago

How is this any better than what you'll get from Google? Now that the cat's out of the bag, a lot of results may even be generated