3 ms·

What's a sample prompt that you've used? Every time I've tried to use one for programming, they invent APIs that don't exist (but sound like they might) or fail

What's a sample prompt that you've used? Every time I've tried to use one for programming, they invent APIs that don't exist (but sound like they might) or fail to produce something that does what it says it does.

GaggiX 2y ago
Have you tried Claude 3.5 Sonnet?
disgruntledphd2 2y ago
Use Python or JS. The models definitely don't seem to perform as well on less hyper prevalent languages.
- randomdata 2y ago
  Even then it is hit and miss. If you are doing something that is also copy/paste-able out of a StackOverflow comment, you're apt to be fine, but as soon as you are doing anything slightly less common... Good luck.
  - disgruntledphd2 2y ago
    Yeah, fair. It's good for short snippets and ways of approaching the problem but not great at execution. It's like infinitely tailored blog posts, for me at least.
    - randomdata 2y ago
      True. It can be good at giving you pointers towards approaching the problem, even if the result is flawed, for slightly less common problems. But as you slide even father towards esotericism, there is no hope. It won't even get you in the right direction. Unfortunately – as that is where it would be most useful.
brianshaler 2y ago
No matter the prompt, there's a significant difference between how it handles common problems in popular languages (python, JS) versus esoteric algorithms in niche languages or tools. I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.
pdntspa 2y ago
I just ask it for what I want in very specific detail, stating the language and frameworks in use. I keep the ideas self-contained -- for example if I need something for the frontend I will ask it to make me a webcomponent. Asking it to not make assumptions and ask questions on ambiguities is also very helpful. It tends to fall apart on bigger asks with larger context. Breaking your task into discrete subtasks works well.

GaggiX 2y ago

Have you tried Claude 3.5 Sonnet?

disgruntledphd2 2y ago

Use Python or JS. The models definitely don't seem to perform as well on less hyper prevalent languages.

randomdata 2y ago
Even then it is hit and miss. If you are doing something that is also copy/paste-able out of a StackOverflow comment, you're apt to be fine, but as soon as you are doing anything slightly less common... Good luck.
- disgruntledphd2 2y ago
  Yeah, fair. It's good for short snippets and ways of approaching the problem but not great at execution. It's like infinitely tailored blog posts, for me at least.
  - randomdata 2y ago
    True. It can be good at giving you pointers towards approaching the problem, even if the result is flawed, for slightly less common problems. But as you slide even father towards esotericism, there is no hope. It won't even get you in the right direction. Unfortunately – as that is where it would be most useful.

brianshaler 2y ago

No matter the prompt, there's a significant difference between how it handles common problems in popular languages (python, JS) versus esoteric algorithms in niche languages or tools. I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.

pdntspa 2y ago

I just ask it for what I want in very specific detail, stating the language and frameworks in use. I keep the ideas self-contained -- for example if I need something for the frontend I will ask it to make me a webcomponent. Asking it to not make assumptions and ask questions on ambiguities is also very helpful. It tends to fall apart on bigger asks with larger context. Breaking your task into discrete subtasks works well.