4 ms·
Where the GAI hallucinates an api it is not always easy to find out if it exists given different versions and libraries for a given task, it can easily waste 10
by grey-area 2y ago
Where the GAI hallucinates an api it is not always easy to find out if it exists given different versions and libraries for a given task, it can easily waste 10 mins trying to find the promised api, particularly when search results also include generated ai answers.
Also there are plenty of mistakes that will compile and give subtle errors, particularly in dynamic languages and those which allow implicit coercion. Javascript comes to mind. The code can easily be runnable but wrong as well (or worse inconsistent and confusing) and this does happen in practice.
- simonw 2y agoMy blog post is all about the fact that "code can easily be runnable but wrong as well" - THAT's the thing you need to worry about, not hallucinated methods.
- margalabargala 2y agoA hallucinated API that is not near-trivially verifiable, is wishful thinking on the part of the AI almost every time.
- CSSer 2y agoMy favorite is when it hallucinates a library that does exist and ostensibly has a relevant title and methods but upon investigation proves to be for a purpose wholly unrelated to the current task. To make matters worse, this discovery will be greeted, every time, by "Ah, you're right!"
- deleted 2y ago[deleted]
- simonw 2y agoYeah, those are pretty frustrating. I've learned not to ask the question "Does library X have feature Y?" because if that feature sounds like a good idea it'll often confidently tell me that it exists when it doesn't.