4 ms·
Question-answering is something Google pivoted toward with great enthusiasm but never quite nailed down. They'd sometimes get some questions right, but it was
by marginalia_nu 3y ago
Question-answering is something Google pivoted toward with great enthusiasm but never quite nailed down. They'd sometimes get some questions right, but it was more of a broken clock sort of a deal.
- kristopolous 3y agoMost implementations of this have a race towards generalities. The biggest problem used to be when seemingly the whole internet was satisfied with an answer that is extremely wrong and broken when you do it. Chatgpt can work though this without getting into a weird markov cycle maybe half the time which is great. Patterns like "Hey I tried that. It still doesn't work, can you give me another option"
- marginalia_nu 3y agoChatGPT has other failure modes. When a question doesn't have an answer written down somewhere, it really struggles. A case is something like "how do I write a parquet file in Java without using Hadoop". This not at all trivial but quite possible[1], but ChatGPT will in 100% of the time either hallucinate APIs, disregard the instructions to not use Hadoop or give otherwise plausible but incorrect-looking answers. The trick is that it isn't doable by simply finding the correct dependencies and API calls, you need extract and override filesystem classes from the Hadoop project to cut those ties. [1] https://github.com/strategicblue/parquet-floor https://github.com/strategicblue/parquet-floor
- kristopolous 3y agoYou can call it out "hey you just made that up. Think hard and give me a real answer" I don't know if "think hard" does anything but it seems to work and if I was the one making chatgpt I'd certainly have configurable keywords like that to tweak the generation settings - mostly so I could skate by on cheaper queries 90+% of the time and then have a fix when they fail