3 ms·

A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs

31 points by rntn 2y ago

Mathnerd314 2y ago
> One course of action that we chose not to pursue for ethical reasons was publishing actual packages using hallucinated package names to PyPI I mean, this makes sense from a security perspective. But from a language usage perspective, if there is a missing package that would be super-useful, then implementing and publishing that package would be a win. I'm curious what the package names were, they seem to have deliberately omitted any package names. Maybe there are some good package ideas in the 19% of names that were hallucinated by multiple models.
- thecrash 2y ago
  By mistakenly declaring the existence of certain packages at scale, the model causes those packages to be created and published. What initially seemed like a hallucination was in fact hyperstition...
  - Terr_ 2y ago
    > hyperstition Not to be confused with substition. > 71-hour Ahmed was not superstitious. He was substitious, which put him in a minority among humans. He didn't believe in the things everyone believed in but which nevertheless weren't true. He believed instead in the things that were true in which no one else believed. There are many such substitions, ranging from ‘It’ll get better if you don’t pick at it’ all the way up to ‘Sometimes things just happen.’ -- Jingo by Terry Pratchett
- night862 2y ago
  No please don’t spam the repo with ai trash aliases!!! The correct and idiomatic way to implement this is to redirect the insane ai guessing to a local proxy which can perform the required search and replace, if that truly is all you need (it is not)
- Mountain_Skies 2y ago
  It would be useful for Software Composition Analysis tools to know the list so they can be flagged. But that can happen on a direct basis instead of publishing the list.
SirMaster 2y ago
This seems to be a problem I frequently run into. The LLM tends to suggest using libraries that don't seem to ever have existed as far as I can find.
- IncreasePosts 2y ago
  Just ask the program to implement those packages too? If the packages don't exist, maybe they should exist.
  - tovej 2y ago
    I hope you're joking
- emptiestplace 2y ago
  I always specify the libraries I want it to use. If I'm not sure, or it introduces something I'm not familiar with, I spend a few minutes comparing similar libraries so I can.

31 points by rntn 2y ago

> One course of action that we chose not to pursue for ethical reasons was publishing actual packages using hallucinated package names to PyPI I mean, this makes sense from a security perspective. But from a language usage perspective, if there is a missing package that would be super-useful, then implementing and publishing that package would be a win. I'm curious what the package names were, they seem to have deliberately omitted any package names. Maybe there are some good package ideas in the 19% of names that were hallucinated by multiple models.

thecrash 2y ago
By mistakenly declaring the existence of certain packages at scale, the model causes those packages to be created and published. What initially seemed like a hallucination was in fact hyperstition...
- Terr_ 2y ago
  > hyperstition Not to be confused with substition. > 71-hour Ahmed was not superstitious. He was substitious, which put him in a minority among humans. He didn't believe in the things everyone believed in but which nevertheless weren't true. He believed instead in the things that were true in which no one else believed. There are many such substitions, ranging from ‘It’ll get better if you don’t pick at it’ all the way up to ‘Sometimes things just happen.’ -- Jingo by Terry Pratchett
night862 2y ago
No please don’t spam the repo with ai trash aliases!!! The correct and idiomatic way to implement this is to redirect the insane ai guessing to a local proxy which can perform the required search and replace, if that truly is all you need (it is not)
Mountain_Skies 2y ago
It would be useful for Software Composition Analysis tools to know the list so they can be flagged. But that can happen on a direct basis instead of publishing the list.

SirMaster 2y ago

This seems to be a problem I frequently run into. The LLM tends to suggest using libraries that don't seem to ever have existed as far as I can find.

IncreasePosts 2y ago
Just ask the program to implement those packages too? If the packages don't exist, maybe they should exist.
- tovej 2y ago
  I hope you're joking
emptiestplace 2y ago
I always specify the libraries I want it to use. If I'm not sure, or it introduces something I'm not familiar with, I spend a few minutes comparing similar libraries so I can.