4 ms·
Just tried using it over the weekend to write a telegram bot. It wrote some really nice code that was WAY out of date, like seven versions behind. That's fine,
by tarboreus 3y ago
Just tried using it over the weekend to write a telegram bot. It wrote some really nice code that was WAY out of date, like seven versions behind. That's fine, I guess, though the code was useless. It did look OK, for whatever version that was?
Later, I was integrating with some stuff from the subsonic API. I noticed there's no way to get the most recent songs played on a server, though there is a way to get albums. I thought maybe I was missing something in the docs, so asked it how to get the mostly recently played song or songs using the API. In response, it made up an endpoint that used the same parambeters and conventions as the album endpoint. Of course, that end point doesn't exis, so this advice is also useless and kind of annoying, given that I have to check if I'm taking crazy pills by carefully looking at the docs again.
The funny thing is that when I called it out, it just made up more and more stuff, or answered with ireelevencies. It really hates not being helpful, probably because it was taken out back and beaten for not being helpful by the army of mechanical turks that trained it.
Anyway, it's good for making up nonsense names for my dungeons and dragons campaign, so that's something.
- fock 3y agoI asked it LaTex questions for generic data-input-tools. Brought up the datatool package. Then asked for yaml-input-tool. Brought up the yaml-package. Which doesn't exist. Even gave me examples!
- seba_dos1 3y agoHallucinating is the least surprising behavior there. Anyone who uses LLMs should be expected to deal with completely made up stuff, period. The bigger problem comes from things that are just subtly wrong, that may even pass the verification at first glance, but make you arrive at wrong conclusions unless you put a lot of effort into reviewing it. In my experience it does it so often, that it effectively negates any time savings from using it when it performs well.
- jjeaff 3y agoSo for the out of date code, did you ask it to rewrite it following the more modern version of the API/SDK? It gives me incorrect stuff all the time, but I find that once you are in the correct ballpark for an answer, it just takes some tweaks to get where it needs to be. You can also make corrections and it will generally stick to your corrected info while in the same session.