4 ms·
One thing it can't really do is admitting that it doesn't actually know stuff. It's similar to creative pupil that asked specific question during lesson doesn't
by scotty79 4y ago
One thing it can't really do is admitting that it doesn't actually know stuff. It's similar to creative pupil that asked specific question during lesson doesn't actually know the answer and just smoothly makes stuff up as they go.
I asked it how to destructure an iterator in rust. I got answer, with code examples and all. Which of course don't work because you can't directly destructure iterator in rust. You pretty much have to collect it into a Vec and the get slice of it and even then you need additional if to cover the case if it's wrong size.
So I pointed out to AI that you can't do it like that. To which it responded that I'm correct that you can't and even gave correct reasons why. And then proceeded with another confident explanation how you can do it. Also wrong, just slightly less wrong but also bit more off-topic.
Btw when at some point in my career I was helping with vetting potential software developer employees. That was one of the worst qualities a person might have. I was fine with people not knowing something and admitting to that. But I was absolutely scared of people who made stuff up when they didn't know.
My friend just played with it a bit too. Asked it to tell "Hello" in Japanese. Then asked how it is written in Katakana. Then AI responded wrongly and when my friend pointed out that it's actually hiragana it politely agreed.
It's a bit smarter now than it was, because when I asked it to teach me how to add numbers it produced reasonable explanation with correct examples which when I tried it last year resulted in equally confident garbage, but it's just that boundary shifted. The underlying problem of producing wrong answers instead being innately aware of limits of its knowledge still remains.
People touting AI as replacement for google search are still way off. It's great for making stuff up but might look good for simple queries but where it fails completely is on the boundaries of its knowledge because there's no indication that there's a boundary.
- amilios 4y agoThe "knowing what it doesn't know" problem is trickier even than it might seem, too: right now we don't even know how what it does know is encoded in its neurons, other than some vague notion that the knowledge is somehow "distributed" across a ton of distinct neurons (there is no specific neuron/set of neurons that are dedicated to memorizing X or Y fact), so figuring out what it doesn't is not trivial at all
- joe_the_user 4y agoOne thing it can't really do is admitting that it doesn't actually know stuff That's because it "doesn't understand knowing". It's ultimately a search engine. When a smart, wise person knows something, they have something of a model of the limitations knowledge - indeed a serious person's model of their knowledge increases along with their knowledge. And certainly some people don't have anything like a model of knowledge either. But the strength of humans is when they do have this.
- scotty79 4y ago> It's ultimately a search engine. Not even that. It's make stuff up engine. Search engines are able to return "no results found". Current incarnation of GPT chat can tell you it doesn't know something, like current net worth of Elon Musk, but it's definitely canned answer that crudely patches the underlying model response which is never empty.
- quenix 4y agoI'm not convinced that it's a canned response.
- scotty79 4y agoTry asking it about anything about real life people. You'll often encounter a sentence or two of agenerated test but then very similarily phrased excuse why it can't know any better as the "shepard" however it is implemented kicks in.
- throwaway2037 4y agoI like your point about interviewing programmers. I do find the candidates who really standout, will admit they don't know the answer, but either ask for a hint, or ask if they can guess. Those rabbit holes are always the most interesting during an interview.