4 ms·

I tried GPT-3 and it generated code examples that just ~made up~ some library that didn't exist at all. It had code that in theory would have ran, if that libra

I tried GPT-3 and it generated code examples that just ~made up~ some library that didn't exist at all. It had code that in theory would have ran, if that library ever existed at all, but it didn't, and it consistently came up with code examples using this imaginary library. I thought that wasn't very impressive. What I'm finding it's extremely great for is writing drivel. I'm talking ecommerce product descriptions, product names, copy... it's really awesome for making one of my side gigs less of a chore.

ianbicking 4y ago
I asked it to write some GPT-related code and it used a gpt3 library (which does not exist) instead of the openai library. I was amused by the self-blindness. (Forgot to actually look it up... seems like it does exist but shouldn't, as it's an empty placeholder: https://pypi.org/project/gpt3/ https://pypi.org/project/gpt3/)
- jamal-kumar 4y ago
  It's important to realize it's just writing out the next thing that seems most likely given the input. I was pretty surprised playing around with github copilot that it could understand the context of a CSV file and filled in all the headers and stuff when I asked it to, then I realized it could probably just arbitrarily read anything on my computer and promptly uninstalled it.
  - 0-_-0 4y ago
    It only reads your source code
  - holoduke 4y ago
    It doesn't get out of your project. And copilot is a trustworthy entity.
thefourthchime 4y ago
> it generated code examples that just ~made up~ some library that didn't exist at all Just ask GPT-3 for the code in the library, what's the problem? /s
- bilsbie 4y ago
  Or dependency injection.
  - sereguze 4y ago
    —> do it without external libraries —> do it this specific way —> damn it I’ll code it myself
- alar44 4y ago
  No, really. This is how I use it. Get it to emulate a Linux terminal. Seed the history with ls / -lah, lscpu, lsb_release -a etc. Then at that point {use git to install a Python project that is a Slackbot template}. Boom, it clones the fake library. cd slack-bot-template, cat main.py and away you go. {Modify the project to do X and commit} Etc etc.
jamesjyu 4y ago
Yes! I've been calling this Hallucinated Scripts. More examples here: https://twitter.com/jamesjyu/status/1600959843669377025 https://twitter.com/jamesjyu/status/1600959843669377025
motoxpro 4y ago
It trips me out that people still try to get GPT3 to do deterministic things (math, cite a source, recite dates, etc). Not sure if its a lack of creativity or just a stubborn view of what is valueable.
- sereguze 4y ago
  I tried because I didn’t know better but eventually figured it out and just asked “can you even do inference” the answer being no
  - int_19h 4y ago
    I hope you didn't ask ChatGPT itself, because it will quite happily lie about things that it doesn't or cannot do.
- jamal-kumar 4y ago
  It would be neat if it COULD do those things, though, maybe by hitting something closer to wolfram alpha when it determ... wait Yeah you have a point. It does boldly claim that it's good at code, though I find a lot more utility just searching github and gists.

ianbicking 4y ago

I asked it to write some GPT-related code and it used a gpt3 library (which does not exist) instead of the openai library. I was amused by the self-blindness. (Forgot to actually look it up... seems like it does exist but shouldn't, as it's an empty placeholder: https://pypi.org/project/gpt3/ https://pypi.org/project/gpt3/)

jamal-kumar 4y ago
It's important to realize it's just writing out the next thing that seems most likely given the input. I was pretty surprised playing around with github copilot that it could understand the context of a CSV file and filled in all the headers and stuff when I asked it to, then I realized it could probably just arbitrarily read anything on my computer and promptly uninstalled it.
- 0-_-0 4y ago
  It only reads your source code
- holoduke 4y ago
  It doesn't get out of your project. And copilot is a trustworthy entity.

thefourthchime 4y ago

> it generated code examples that just ~made up~ some library that didn't exist at all Just ask GPT-3 for the code in the library, what's the problem? /s

bilsbie 4y ago
Or dependency injection.
- sereguze 4y ago
  —> do it without external libraries —> do it this specific way —> damn it I’ll code it myself
alar44 4y ago
No, really. This is how I use it. Get it to emulate a Linux terminal. Seed the history with ls / -lah, lscpu, lsb_release -a etc. Then at that point {use git to install a Python project that is a Slackbot template}. Boom, it clones the fake library. cd slack-bot-template, cat main.py and away you go. {Modify the project to do X and commit} Etc etc.

jamesjyu 4y ago

Yes! I've been calling this Hallucinated Scripts. More examples here: https://twitter.com/jamesjyu/status/1600959843669377025 https://twitter.com/jamesjyu/status/1600959843669377025

motoxpro 4y ago

It trips me out that people still try to get GPT3 to do deterministic things (math, cite a source, recite dates, etc). Not sure if its a lack of creativity or just a stubborn view of what is valueable.

sereguze 4y ago
I tried because I didn’t know better but eventually figured it out and just asked “can you even do inference” the answer being no
- int_19h 4y ago
  I hope you didn't ask ChatGPT itself, because it will quite happily lie about things that it doesn't or cannot do.
jamal-kumar 4y ago
It would be neat if it COULD do those things, though, maybe by hitting something closer to wolfram alpha when it determ... wait Yeah you have a point. It does boldly claim that it's good at code, though I find a lot more utility just searching github and gists.