4 ms·
I think the thing that is most needed now, is some way to evaluate and ensure that the results being returned are accurate. Just providing an answer isn't enoug
by yurishimo 4y ago
I think the thing that is most needed now, is some way to evaluate and ensure that the results being returned are accurate. Just providing an answer isn't enough. Anyone can ask me a question, and I can give you an answer, but for most topics, it probably won't be the correct answer, but if I know enough buzzwords, I might be able to string together something that _sounds like it could be_ a correct answer.
We've seen this already. Except in limited cases, ChatGPT will not tell you that it doesn't know the answer. Or it will confidently give you the wrong answer. I'm reminded of a thread I saw last week where chat GPT was certain that 42 was a smaller number than 65(?) and when asked "can you count from 65 to 42?", it kept going up and up until it inevitably hit some sort of character count limit imposed by the service's algorithm for generating responses.
The cool part IMO about ChatGPT is the back and forth conversational style of sharing information. Now we need to integrate that with traditional search indexes and also work to provide the "AI" some sort of score to indicate if the returned results are accurate and be more upfront with the user if a certain confidence interval is not reached. An example of this in traditional search tools, is the "did you mean xxx?" prompt.
Until we can "trust" the results to be accurate, I don't see this technology being widely usable to expand the capabilities of users past their own knowledge. If you cannot easily verify the results as they are given to you today, then I don't really see the utility of the program beyond novelty.
- datavirtue 4y agoI used it to find quick answers to congressional procedures and laws relating to elections. Very quick, informative replies that would have taken forever to unearth using Google.
- Spivak 4y agoI also found this useful, I was able to get an answer to "is this specific thing in this situation legal in this state" and have it produce correct references to the state code. I don't actually care if it produces an incorrect answer or summary (it wasn't but that's irrelevant), the value is being able to take some complicated context that a search engine can't understand and say "look right here."
- SketchySeaBeast 4y agoDid you double check the results? I know, it's no longer quick then, but you can't be sure that it didn't straight up lie to you.