3 ms·
> And yet people are doing that to LLMs. "Haha, look at that LLM hallucinating", where half the time the prompt is misleading or wrong. I mean... I ask it to g
by maximus-decimus 3y ago
> And yet people are doing that to LLMs. "Haha, look at that LLM hallucinating", where half the time the prompt is misleading or wrong.
I mean... I ask it to give me Python code and it gives me code that doesn't work and makes up libraries that don't actually exist. That's not a prompt problem. A human would understand there's an implicit "code that actually works and isn't pulled out of your ass".
- harimau777 3y agoCould that be similar to how most people have a few bugs in the first draft of their code; particularly when writing from memory (e.g. during a whiteboard interview)? For example, I always have to lookup whether the function is Array.every or Array.some in JavaScript. That doesn't seem too different than making up a library to me.
- _ea1k 3y agoIf you make the human write the code on a whiteboard without a python interpreter, they are likely to occasionally make up libraries that don't exist. The LLM is operating similarly in a lot of cases. It doesn't have an IDE with intellisense acting as an immediate feedback mechanism.
- SirMaster 3y agoBut when you ask the LLM where you can read about the library it's referencing (that it made up), it then makes up a URL for it that has never existed, (at least according to the internet archive). A human certainly doesn't do that in my experience. Apparently LLMs know when they are making stuff up (or so I keep getting told that over and over by others here). But clearly they are terrible at communicating about when they are making things up which is a big problem to me. I wish they would just say when they don't know something rather than make up BS. Plenty of humans are perfectly capable of acknowledging when they don't know something.