3 ms·
> All the code generation tools, StarCoder included, still have hallucinations. This also includes humans. We "hallucinate" in very similar ways. For example m
by theaiquestion 3y ago
> All the code generation tools, StarCoder included, still have hallucinations.
This also includes humans. We "hallucinate" in very similar ways. For example mistaking localhost:8080 for localhost:8008 in a large config file. Attempting to use methods that were deprecated and no longer exist, etc.
IMO there's two ways to prevent this is - one is to make better performing models (architecture/training data/training amount/etc)
The other is the exact same as humans. Compile time tools that let it know immediately if it hallucinated, types, linting, tests, etc.
You just do it as a loop the exact same as a human. You write code, the compiler tells you that method doesn't exist, you adjust your code/consult the documents (also doable with agents).