5 ms·

This is similar to how I’ve been using ChatGPT. If you know what you need to do and how to do it, then you can tell ChatGPT what to do in a descriptive prompt.

This is similar to how I’ve been using ChatGPT. If you know what you need to do and how to do it, then you can tell ChatGPT what to do in a descriptive prompt. It gets you there 75% of the way and then you need to tweak things or ask it to refactor. For me, it’s removed a lot of grunt work and I can focus on the problem rather than coding. It reminds me of working with a junior developer, you need to understand how to communicate with them effectively.

camillomiller 4y ago
Same experience. For me the golden standard for a problem I can conceptualize but I can’t write code for is to start a prompt with “how would you do this with —given language”? That has taught me some things I didn’t know and helped me formalize and understand concepts better (I think). Also, on a more funny note, it’s the best ffmpeg interface I’ve ever used :)
- spookthesunset 4y ago
  Yes. I love the idea of using it for ffmpeg!
- moffkalast 4y ago
  Yeah it's starting to remind me of the Star Trek holodeck interface. You still need to know exactly what needs to be done, but it can handle the boilerplate and a rough first implementation, then I can spend my time tweaking it a bit to get exactly what's needed. Probably saved me a whole day of work on a project yesterday.
- spaceman_2020 4y ago
  For some problems, somehow GPT-3.5 is better than GPT-4. I asked it to help me extract certain values from a spreadsheet. It gave me a complicated and overly long formula using FIND and MID that didn't even work. GPT-3.5, otoh, gave me a neat little REGEXEXTRACT formula that worked perfectly. When I pointed out to GPT-4 that it can just use regex instead, it "apologized" and again rewrote a formula that didn't work.
avereveard 4y ago
yeah people are scratching the surface of the potential. you can write prompt to clean unstructured text into structured text. that becomes the work area. instead of asking gpt "write me some code in pyton that does x" you can just tell gtp "do x with the structured data in the work area and present a structured result" gpt becomes the computation engine granted it's super expensive and quite slow, today but with a few plugins to take care of mechanical things and the token memory space becoming increasingly bigger, writing the programming step directly on there may be the standard. as soon as vector storage and reasoning steps gets well integrated, it's going explode.
- deleted 4y ago
  [deleted]
- ZitchDog 4y ago
  I doubt it will be the standard. I think it will always be faster to process data using a script generated by the LLM. One thing I could see is a "self updating" script, that adjusts its logic on the fly when new data are found.
  - avereveard 4y ago
    on one hand it's also explotaible as hell via prompt injection, on the other hand for certain tasks is more robust because it works off the intention you give it, not the instructions, so it can get back on track.
colordrops 4y ago
A form of literate code could be to add the prompt as a comment in the source code, along with a version and seed (like stable diffusion), plus the diff from the output to get what you need. Unfortunately the gpt api is non deterministic, probably due to stupid censorship requirements.
louiskw 4y ago
> If you know what you need to do and how to do it Having used it for the last week or so to help me translate Python -> Rust, I'd say GPT4 is on par with even senior engineers for anything built in. The key areas it falls down on are anything involving org/repo knowledge or open source where things can be out of date. There's a few teams working on intelligent org/repo/opensource context fetching, which I think will lead to another 10X improvement.
- logankeenan 4y ago
  > I'd say GPT4 is on par with even senior engineers I agree, I avoided saying that in my original comment to avoid nit picking. The depth of knowledge is incredible. It’ll just make junior level mistakes while writing senior level code. > The key areas it falls down on are anything involving org/repo knowledge or open source where things can be out of date. With GPT-4, I’ll paste snippets of code after the prompt so it has that context or even documentation of the API I need it to work with. The web client is still limited in the amount of tokens it’ll accept in the prompt, but that is expected to increase at some point
  - IanCal 4y ago
    Also the setup described in the post is like pairing with someone that can't run the code and is writing in notepad. More intelligently putting it in a loop & other helpers for it (which we're now seeing worked on more) should massively improve performance.
- f0e4c2f7 4y ago
  This is also available with chatgpt plugins in alpha, one of the plugins is websearch. It works pretty well in bing which also allows this and is reportedly based on gpt4. Short of that, another thing I've found to work pretty well is pasting in the api docs for the relevant api at the beginning of the prompt and explaining that is the more updated info.
- bradgessler 4y ago
  This is a use case I could see GPT really excelling—porting libraries from one programming language to another. How awesome would that be? Newish ecosystems like Rust, Crystal, Elixir, etc. could have a larger set of libraries in a shorter amount of time. There’s still a lot of need for a human to be involved, buts it’s not hard to see how this will make individual developers more productive, especially as LLMs mature, plug themselves into interpreters, and become more capable.
js2 4y ago
The other day I made a change to one of our iOS builds such that the ipa (a zip file) now contained a subdirectory called "Symbols/". I wanted to know how much space the compressed symbols grew the overall ipa by. I immediately typed this on the command line: zipinfo -l foo.ipa "Symbols/*" | awk '{t+=$6}END{print t/1024/1024}' I work with developers who are not CLI-fluent and wondered what they'd do. And then I realized, they might ask ChatGPT. So I asked it: "How do I figure out how much space a subdirectory of files takes up within a zip file?" It took a lot of prompting to guide it to the correct answer: https://imgur.com/a/kAaMova https://imgur.com/a/kAaMova The weird part to me is how sure it is of itself even when it's completely wrong. (Not too different from humans I work with, self included, but still...) e.g. > "The reason I moved the subdir filtering to 'awk' is that the 'zipinfo' command doesn't support filtering by subdirectory like the 'unzip' command does." Where does it come up with that assertion and why is it so sure? Then you just tell it it's wrong and it apologizes and moves on. Now, the addendum here is that if either of us had read the zipinfo man page more closely, we would've realized it's even simpler: zipinfo -lt foo.ipa "Symbols/*" | tail -1 --- I also tried to use it to translate about 10 lines of Ruby that was just mapping a bit of json from one shape to another to jq. It really went off into the weeds on that one and initially told me it was impossible because the Ruby was opening a file and jq has no I/O. So I told it to ignore that part and assume stdin and stdout and it got closer, but still came up with something pretty weird. The core of the jq was pretty close, but for example, it did this to iterate over an object's values: to_entries | .value.foo Instead of just using the object value iterator: [].foo --- One of my pet peeves is developers who never bother to learn idiomatic code for a particular language, and both of these turn out to be that sort of thing. I guess it's no worse than folks who copy/paste from StackOverflow, but I kind of hoped ChatGPT would generate things that are closer to idiomatic and more based on reference documentation but I haven't seen that to be the case yet. --- Has anyone here tried using it for code reviews instead of writing code in the first place? How does it do with that?
- adwf 4y ago
  I think part of the problem is that they've wrapped all the responses in a positive language framing to make it feel more human. It's always "Sure thing!" or "Yeah I can do that!", and it responds so quickly that you equate it to when a human responds instantly with a strong affirmative - you assume they're really confident and you're getting the right result. If it took 5-10s on each query, had no positive framing and provided some sort of confidence score for each answer, I think we'd have different expectations of what it can do.
cameronfraser 4y ago
Exactly! I've been telling people for a bit, it's like having a junior dev on the team that knows a lot of weird stuff
brundolf 4y ago
Maybe I should just try it, but I don't feel like there are many situations where the process of "come up with a way to describe exactly what I want, generate it, and then debug the output until it works" would be faster than "just write it myself". And most of the ones I can think of would be in a situation where I'm working in an unfamiliar language/framework, and it's less about not having to type out the code and more about not having to context-switch to reference the docs for something I don't know off the top of my head Which- maybe that's just the thing, maybe that's what people are using this for right now (This could change once it gets to the point of stamping out entire projects, but for now)
- CookieCrisp 4y ago
  If you haven't tried it out, I recommend you do. I've been blown away with how much it will just get right. I started out giving it small tasks, but, I have continually been in a loop of "oh, I could have just asked it to do this larger task that contained the subtask I asked for, and it would have done it", because I keep thinking "surely it wont be able to do the whole thing", and each iteration I am asking it for more and more. As far as debugging, I find that I can just say "are there any bugs in that code?" and it does a great job at finding them if they're there (even though it's what it just gave me)
- montecarl 4y ago
  I'll give two examples of code I could for sure write myself but that I had gpt4 write for me because I figured it would be faster (and it was). The first one was to write a python script to watch a list of files given as cmd line arguments and plot their output anytime one of the files changed. It wrote a 100 line python script with nice argument parsing to include several options (like title and axes labels). It has one tiny bug in it that took a couple of minutes to fix. When I pointed out the bug it was able to fix it itself (had to do with comparing relative to abs file paths). If I wrote the script myself I would not have made something general purpose and it would have taken maybe 30 minutes to do. The second example required no fixing and appears bug free. I asked it to write a python function to take a time trace, calculate and plot the FFT, then apply FFT low pass filtering, and then also plot the filtered time signal vs the original. This is all straight forward numpy code but I don't work with FFTs often and would have had to lookup a bunch of different API docs. Way faster. I have also had it write some c macros for me, since complex C macros can be hard to escape properly and I'm not comfortable with the syntax. Its 100% successful there.
dimal 4y ago
Same. It’s like working with an incredibly smart junior engineer who doesn’t think things through, sometimes solves problems that don’t exist, and then when you ask for a minor change, inexplicably rewrites everything to be “better”. I haven’t gotten great results yet, but it feels like I just have to figure out how to ask questions correctly so I can lead it down the right path. Seems like I’ve been asking it to solve problems that are too large, and once there’s an interaction of two or more functions, it goes off the rails. But if I define the overall structure and the interfaces, then I have better luck getting it to fill in the functions. Our jobs are safe, for now.
TheHappyOddish 4y ago
That's precisely how I frame it to people. I'm a member of the management team for a company, and everyone was very excited for ChatGPT. As someone with a technical background, the caveats were a lot more obvious to me from the start. Treating it as an incredibly fast, eager junior developer who is desperate to please and submits code without testing it has made it a lot more usable.

camillomiller 4y ago

Same experience. For me the golden standard for a problem I can conceptualize but I can’t write code for is to start a prompt with “how would you do this with —given language”? That has taught me some things I didn’t know and helped me formalize and understand concepts better (I think). Also, on a more funny note, it’s the best ffmpeg interface I’ve ever used :)

spookthesunset 4y ago
Yes. I love the idea of using it for ffmpeg!
moffkalast 4y ago
Yeah it's starting to remind me of the Star Trek holodeck interface. You still need to know exactly what needs to be done, but it can handle the boilerplate and a rough first implementation, then I can spend my time tweaking it a bit to get exactly what's needed. Probably saved me a whole day of work on a project yesterday.
spaceman_2020 4y ago
For some problems, somehow GPT-3.5 is better than GPT-4. I asked it to help me extract certain values from a spreadsheet. It gave me a complicated and overly long formula using FIND and MID that didn't even work. GPT-3.5, otoh, gave me a neat little REGEXEXTRACT formula that worked perfectly. When I pointed out to GPT-4 that it can just use regex instead, it "apologized" and again rewrote a formula that didn't work.

avereveard 4y ago

yeah people are scratching the surface of the potential. you can write prompt to clean unstructured text into structured text. that becomes the work area. instead of asking gpt "write me some code in pyton that does x" you can just tell gtp "do x with the structured data in the work area and present a structured result" gpt becomes the computation engine granted it's super expensive and quite slow, today but with a few plugins to take care of mechanical things and the token memory space becoming increasingly bigger, writing the programming step directly on there may be the standard. as soon as vector storage and reasoning steps gets well integrated, it's going explode.

deleted 4y ago
[deleted]
ZitchDog 4y ago
I doubt it will be the standard. I think it will always be faster to process data using a script generated by the LLM. One thing I could see is a "self updating" script, that adjusts its logic on the fly when new data are found.
- avereveard 4y ago
  on one hand it's also explotaible as hell via prompt injection, on the other hand for certain tasks is more robust because it works off the intention you give it, not the instructions, so it can get back on track.

colordrops 4y ago

A form of literate code could be to add the prompt as a comment in the source code, along with a version and seed (like stable diffusion), plus the diff from the output to get what you need. Unfortunately the gpt api is non deterministic, probably due to stupid censorship requirements.

louiskw 4y ago

> If you know what you need to do and how to do it Having used it for the last week or so to help me translate Python -> Rust, I'd say GPT4 is on par with even senior engineers for anything built in. The key areas it falls down on are anything involving org/repo knowledge or open source where things can be out of date. There's a few teams working on intelligent org/repo/opensource context fetching, which I think will lead to another 10X improvement.

logankeenan 4y ago
> I'd say GPT4 is on par with even senior engineers I agree, I avoided saying that in my original comment to avoid nit picking. The depth of knowledge is incredible. It’ll just make junior level mistakes while writing senior level code. > The key areas it falls down on are anything involving org/repo knowledge or open source where things can be out of date. With GPT-4, I’ll paste snippets of code after the prompt so it has that context or even documentation of the API I need it to work with. The web client is still limited in the amount of tokens it’ll accept in the prompt, but that is expected to increase at some point
- IanCal 4y ago
  Also the setup described in the post is like pairing with someone that can't run the code and is writing in notepad. More intelligently putting it in a loop & other helpers for it (which we're now seeing worked on more) should massively improve performance.
f0e4c2f7 4y ago
This is also available with chatgpt plugins in alpha, one of the plugins is websearch. It works pretty well in bing which also allows this and is reportedly based on gpt4. Short of that, another thing I've found to work pretty well is pasting in the api docs for the relevant api at the beginning of the prompt and explaining that is the more updated info.
bradgessler 4y ago
This is a use case I could see GPT really excelling—porting libraries from one programming language to another. How awesome would that be? Newish ecosystems like Rust, Crystal, Elixir, etc. could have a larger set of libraries in a shorter amount of time. There’s still a lot of need for a human to be involved, buts it’s not hard to see how this will make individual developers more productive, especially as LLMs mature, plug themselves into interpreters, and become more capable.

js2 4y ago

The other day I made a change to one of our iOS builds such that the ipa (a zip file) now contained a subdirectory called "Symbols/". I wanted to know how much space the compressed symbols grew the overall ipa by. I immediately typed this on the command line: zipinfo -l foo.ipa "Symbols/*" | awk '{t+=$6}END{print t/1024/1024}' I work with developers who are not CLI-fluent and wondered what they'd do. And then I realized, they might ask ChatGPT. So I asked it: "How do I figure out how much space a subdirectory of files takes up within a zip file?" It took a lot of prompting to guide it to the correct answer: https://imgur.com/a/kAaMova https://imgur.com/a/kAaMova The weird part to me is how sure it is of itself even when it's completely wrong. (Not too different from humans I work with, self included, but still...) e.g. > "The reason I moved the subdir filtering to 'awk' is that the 'zipinfo' command doesn't support filtering by subdirectory like the 'unzip' command does." Where does it come up with that assertion and why is it so sure? Then you just tell it it's wrong and it apologizes and moves on. Now, the addendum here is that if either of us had read the zipinfo man page more closely, we would've realized it's even simpler: zipinfo -lt foo.ipa "Symbols/*" | tail -1 --- I also tried to use it to translate about 10 lines of Ruby that was just mapping a bit of json from one shape to another to jq. It really went off into the weeds on that one and initially told me it was impossible because the Ruby was opening a file and jq has no I/O. So I told it to ignore that part and assume stdin and stdout and it got closer, but still came up with something pretty weird. The core of the jq was pretty close, but for example, it did this to iterate over an object's values: to_entries | .value.foo Instead of just using the object value iterator: [].foo --- One of my pet peeves is developers who never bother to learn idiomatic code for a particular language, and both of these turn out to be that sort of thing. I guess it's no worse than folks who copy/paste from StackOverflow, but I kind of hoped ChatGPT would generate things that are closer to idiomatic and more based on reference documentation but I haven't seen that to be the case yet. --- Has anyone here tried using it for code reviews instead of writing code in the first place? How does it do with that?

adwf 4y ago
I think part of the problem is that they've wrapped all the responses in a positive language framing to make it feel more human. It's always "Sure thing!" or "Yeah I can do that!", and it responds so quickly that you equate it to when a human responds instantly with a strong affirmative - you assume they're really confident and you're getting the right result. If it took 5-10s on each query, had no positive framing and provided some sort of confidence score for each answer, I think we'd have different expectations of what it can do.

cameronfraser 4y ago

Exactly! I've been telling people for a bit, it's like having a junior dev on the team that knows a lot of weird stuff

brundolf 4y ago

Maybe I should just try it, but I don't feel like there are many situations where the process of "come up with a way to describe exactly what I want, generate it, and then debug the output until it works" would be faster than "just write it myself". And most of the ones I can think of would be in a situation where I'm working in an unfamiliar language/framework, and it's less about not having to type out the code and more about not having to context-switch to reference the docs for something I don't know off the top of my head Which- maybe that's just the thing, maybe that's what people are using this for right now (This could change once it gets to the point of stamping out entire projects, but for now)

CookieCrisp 4y ago
If you haven't tried it out, I recommend you do. I've been blown away with how much it will just get right. I started out giving it small tasks, but, I have continually been in a loop of "oh, I could have just asked it to do this larger task that contained the subtask I asked for, and it would have done it", because I keep thinking "surely it wont be able to do the whole thing", and each iteration I am asking it for more and more. As far as debugging, I find that I can just say "are there any bugs in that code?" and it does a great job at finding them if they're there (even though it's what it just gave me)
montecarl 4y ago
I'll give two examples of code I could for sure write myself but that I had gpt4 write for me because I figured it would be faster (and it was). The first one was to write a python script to watch a list of files given as cmd line arguments and plot their output anytime one of the files changed. It wrote a 100 line python script with nice argument parsing to include several options (like title and axes labels). It has one tiny bug in it that took a couple of minutes to fix. When I pointed out the bug it was able to fix it itself (had to do with comparing relative to abs file paths). If I wrote the script myself I would not have made something general purpose and it would have taken maybe 30 minutes to do. The second example required no fixing and appears bug free. I asked it to write a python function to take a time trace, calculate and plot the FFT, then apply FFT low pass filtering, and then also plot the filtered time signal vs the original. This is all straight forward numpy code but I don't work with FFTs often and would have had to lookup a bunch of different API docs. Way faster. I have also had it write some c macros for me, since complex C macros can be hard to escape properly and I'm not comfortable with the syntax. Its 100% successful there.

dimal 4y ago

Same. It’s like working with an incredibly smart junior engineer who doesn’t think things through, sometimes solves problems that don’t exist, and then when you ask for a minor change, inexplicably rewrites everything to be “better”. I haven’t gotten great results yet, but it feels like I just have to figure out how to ask questions correctly so I can lead it down the right path. Seems like I’ve been asking it to solve problems that are too large, and once there’s an interaction of two or more functions, it goes off the rails. But if I define the overall structure and the interfaces, then I have better luck getting it to fill in the functions. Our jobs are safe, for now.

TheHappyOddish 4y ago

That's precisely how I frame it to people. I'm a member of the management team for a company, and everyone was very excited for ChatGPT. As someone with a technical background, the caveats were a lot more obvious to me from the start. Treating it as an incredibly fast, eager junior developer who is desperate to please and submits code without testing it has made it a lot more usable.