5 ms·

>I've already run into scenarios where ChatGPT generated code that looked perfectly plausible, except for that the actual API used didn't really exist. Yes! I

>I've already run into scenarios where ChatGPT generated code that looked perfectly plausible, except for that the actual API used didn't really exist. Yes! I remember generating a seemingly reasonable R script except that the library that it called to do most of the work didn't exist! It was like code from an alternate dimension!

PartiallyTyped 4y ago
I have had the same experience with Typescript and Python.
bamboozled 4y ago
I asked it to generate sample aws step function config and as far as I could tell it made up configuration parameters. I know it’s a language model.
Kelteseth 4y ago
Can confirm that this happened to me when I asked ChatGPT to generate a parallax effect in Qt/QML. It simply used a QML Elemened with the name Parallax.
jay-barronville 4y ago
> It was like code from an alternate dimension! Maybe it was? Haha. ChatGPT has seen some things and knows something we don’t!
gattilorenz 4y ago
It’s as if ChatGPT was behaving like a language model with no real connection or understanding of R… hmmmmm…
- alpos 4y ago
  This take strikes me as a little off. Programming languages are language. Unlike natural languages they are also based on context-free grammar. So an understanding of programming languages should actually be easier for even a general language model to incorporate than natural languages. We can expect a bot like this to not really get context clues in natural language, although they seem to be getting better at that, but context is not necessary to have a true and functional understanding of a programming language. That was the point of creating such languages. Using an API that doesn't exist but logically should once the use cases are demonstrated is not an example of lacking understanding, it is an example of advanced insight. A human might have invented the necessary functions inline with the rest of the project but if they are expressing functionality that is commonly applicable, then a common API for those functions is what the humans would eventually converge upon to clean up the code from the initial inline implementation, making it more consistent and readable.
  - layer8 4y ago
    The failure to realize that the API doesn’t exist and therefore the code won’t work in practice, however, is a major lack of insight and understanding.
    - alpos 4y ago
      Agreed. That does seem to be an example of a language model failing to understand the context in which the question was asked. The user was implicitly expecting code that would function when executed immediately and as written with no additional supporting libraries included. This is different from code that would function correctly when executed after having downloaded relevant existing packages. Which is different from code that would function if executed along side additional supporting code from private libraries the user might not have access to. Etc... Yet any of those answers fit for the same prompt, "create an R script that does such and such". The bot's lack of insight is on the likely intention behind the prompt rather than on the requested language. I'd say if it produces any code that fits the syntax and grammatical structure of the requested language, that's enough to say it understands the language.
  - gattilorenz 4y ago
    My answer was mildly tongue in cheek, and I see where you’re going. On the other hand, one of the other posters asked “to generate a parallax effect in Qt/QML. It simply used a QML Elemened with the name Parallax”. Is this an insight, or is this answering “yes, I could” to “could you pass me the salt?”. Maybe the line between the two is a fine one, and I didn’t realize that yet. In general, it feels like copying part of the question (“write parallax code”) in the answer is the easy part of the task…
    - alpos 4y ago
      Saw that, and yea, that's totally a fall-back cop-out type answer. I was pretty much just questioning everyone dismissing the whole category of answers like the R script one. To me there does seem to be some nuance to here that's worth noticing. Some examples of this type of response are indeed too cheap and can be chalked up to lack of training data or something. But in other cases it's actually not immediately obvious whether the answer the user got was their fault for not specifying that they are expecting code that works without additional supporting libraries. A language model can't reasonably be expected to understand an expectation of usability or fitness for purpose in a context the user didn't specify.
VeninVidiaVicii 4y ago
Yeah, a few times when I ask for a reference to something outlandish, it generates a perfectly realistic looking paper alongside a doi link, that’s completely made up. Both the paper and the link link do not exist!
- alpos 4y ago
  I have to wonder, were any of the the hypothesis' in those papers plausibly viable areas of inquiry? Perhaps it could be useful if they train the bot to identify cases like this and state that no such references exist but also provide a thesis or suggest a line of study that would produce such a reference.
  - deleted 4y ago
    [deleted]
dagw 4y ago
I asked if there where any Open Source libraries that implemented a certain algorithm. It gave me links to 3 different GitHub repos, none of which existed.
- euroderf 4y ago
  Did it have the common courtesy to add a comment saying "PRANKED!" ?

PartiallyTyped 4y ago

I have had the same experience with Typescript and Python.

bamboozled 4y ago

I asked it to generate sample aws step function config and as far as I could tell it made up configuration parameters. I know it’s a language model.

Kelteseth 4y ago

Can confirm that this happened to me when I asked ChatGPT to generate a parallax effect in Qt/QML. It simply used a QML Elemened with the name Parallax.

jay-barronville 4y ago

> It was like code from an alternate dimension! Maybe it was? Haha. ChatGPT has seen some things and knows something we don’t!

gattilorenz 4y ago

It’s as if ChatGPT was behaving like a language model with no real connection or understanding of R… hmmmmm…

alpos 4y ago
This take strikes me as a little off. Programming languages are language. Unlike natural languages they are also based on context-free grammar. So an understanding of programming languages should actually be easier for even a general language model to incorporate than natural languages. We can expect a bot like this to not really get context clues in natural language, although they seem to be getting better at that, but context is not necessary to have a true and functional understanding of a programming language. That was the point of creating such languages. Using an API that doesn't exist but logically should once the use cases are demonstrated is not an example of lacking understanding, it is an example of advanced insight. A human might have invented the necessary functions inline with the rest of the project but if they are expressing functionality that is commonly applicable, then a common API for those functions is what the humans would eventually converge upon to clean up the code from the initial inline implementation, making it more consistent and readable.
- layer8 4y ago
  The failure to realize that the API doesn’t exist and therefore the code won’t work in practice, however, is a major lack of insight and understanding.
  - alpos 4y ago
    Agreed. That does seem to be an example of a language model failing to understand the context in which the question was asked. The user was implicitly expecting code that would function when executed immediately and as written with no additional supporting libraries included. This is different from code that would function correctly when executed after having downloaded relevant existing packages. Which is different from code that would function if executed along side additional supporting code from private libraries the user might not have access to. Etc... Yet any of those answers fit for the same prompt, "create an R script that does such and such". The bot's lack of insight is on the likely intention behind the prompt rather than on the requested language. I'd say if it produces any code that fits the syntax and grammatical structure of the requested language, that's enough to say it understands the language.
- gattilorenz 4y ago
  My answer was mildly tongue in cheek, and I see where you’re going. On the other hand, one of the other posters asked “to generate a parallax effect in Qt/QML. It simply used a QML Elemened with the name Parallax”. Is this an insight, or is this answering “yes, I could” to “could you pass me the salt?”. Maybe the line between the two is a fine one, and I didn’t realize that yet. In general, it feels like copying part of the question (“write parallax code”) in the answer is the easy part of the task…
  - alpos 4y ago
    Saw that, and yea, that's totally a fall-back cop-out type answer. I was pretty much just questioning everyone dismissing the whole category of answers like the R script one. To me there does seem to be some nuance to here that's worth noticing. Some examples of this type of response are indeed too cheap and can be chalked up to lack of training data or something. But in other cases it's actually not immediately obvious whether the answer the user got was their fault for not specifying that they are expecting code that works without additional supporting libraries. A language model can't reasonably be expected to understand an expectation of usability or fitness for purpose in a context the user didn't specify.

VeninVidiaVicii 4y ago

Yeah, a few times when I ask for a reference to something outlandish, it generates a perfectly realistic looking paper alongside a doi link, that’s completely made up. Both the paper and the link link do not exist!

alpos 4y ago
I have to wonder, were any of the the hypothesis' in those papers plausibly viable areas of inquiry? Perhaps it could be useful if they train the bot to identify cases like this and state that no such references exist but also provide a thesis or suggest a line of study that would produce such a reference.
- deleted 4y ago
  [deleted]

dagw 4y ago

I asked if there where any Open Source libraries that implemented a certain algorithm. It gave me links to 3 different GitHub repos, none of which existed.

euroderf 4y ago
Did it have the common courtesy to add a comment saying "PRANKED!" ?