5 ms·

ChatGPT with GPT4 has made me much better and faster at solving programming problems, both at work and for working on personal projects. Many people are still

by jstx1 3y ago

ChatGPT with GPT4 has made me much better and faster at solving programming problems, both at work and for working on personal projects. Many people are still sleeping on how useful LLMs are. There's a lot of related things to be skeptical about (big promises, general AI, does it replace jobs, all the new startups that are basically dressed up API calls...) but if you do any kind of knowledge work, there's a good chance that you could it much better if you also used an LLM.

jabradoodle 3y ago
The parent is asking for a specific example use case.
- hayksaakian 3y ago
  I can't speak for OP, but for me I literally never use stack overflow any more, and I spend about 90% less time on Google
  - styfle 3y ago
    Curious if that’s because AI provides better answers? It’s certainly not quicker answers, right?
  - 8organicbits 3y ago
    Are you not fact checking chatgpt? I've seen wrong info, especially subtle things. It seemed reckless to use as-is.
    - blowski 3y ago
      Both ChatGPT and StackOverflow suffer from content becoming outdated. So some highly-upvoted answer on StackOverflow has been out of date since 2011, and now ChatGPT is trained on it. I see the future as writing test cases (perhaps also with ChatGPT), and separately using ChatGPT to write the implementation. Perhaps we will just give it a bunch of test cases and it will return code (or submit a PR) that passes those tests.
    - etimberg 3y ago
      For fun I tried asking chatgpt to create a simple using an opensource project I maintain. The generated answer was sort of correct but not correct enough to copy and paste. It missed including a plugin, used a version of the project that doesn't exist yet, and generated data that wasn't valid datetimes.
  - hallway_monitor 3y ago
    Yep exactly. I guess I haven't hit the 25 messages in 3 hours limit, but whenever there's an API or library I'm not familiar with. I can get my exact example in about 10 seconds from ChatGPT 4
    - iudqnolq 3y ago
      Are those popular APIs? I've found Copilot useful when writing greenfield code, but very unhelpful generating code that uses APIs not popular enough to have significant coverage on StackOverflow. Even if I have examples of correct usage in the same file it still guesses plausible but wrong types. I haven't bought GPT 4 but I'm curious if it's much better at this.
      - lstamour 3y ago
        If you don't mention a library by name it is liable to make something up by picking a popular library in another language and converting the syntax to the language you asked for. If you ask for something impossible in a library it will also frequently make up functions or application settings. If you ask for something obscure but hard to do, it might reply that it's impossible but it is possible if you know how and teach it. I sort of compare prompt engineering to Googling - you sometimes have to search for exactly the right terms that you want to appear in the result in order to get the answer you're looking for. It's just that the flexibility of ChatGPT in writing a direct response sometimes means it will completely make up an answer. There's also a limitation that the web interface doesn't actually let you upload files and has a length limit for inputs. For Copilot, I'm looking forward to Copilot X: https://www.youtube.com/watch?v=3surPGP7_4o https://www.youtube.com/watch?v=3surPGP7_4o
        iudqnolq 3y ago
        This was neither. I've forgotten the exact words I typed but it was something like this. Prompt: fn encode(value: Foo) { capnproto::serialize_packed:: serialize_message(value); } fn decode(input: &[u8]) { Expected: capnproto::serialize_packed:: deserialize_message(input); Generated capnproto::PackedMessageDeserializer::deserialize(input)
- michaelbuckbee 3y ago
  An eye opening example for me was that I was working with a Ruby/Rails class that was testing various IP configurations [1] and I was able to just copy and paste it into chatgpt and say "write some tests for this". It wasn't really anything I couldn't have written in a half hour or so but it was so much faster. The real kicker is that by default chatgpt wrote Rspec and I was able to say "rewrite that in minitest" and it worked. 1 - https://wafris.org/ip-lookup https://wafris.org/ip-lookup
nice_byte 3y ago
what kind of problems are you trying to solve that make gpt-4 so helpful to you?
cbm-vic-20 3y ago
I'm really trying to do the same, for both my work, and personal projects. But the type of answers I need for work (enterprise software, large codebase built over 20+ years) requires a ton of context that I simply cannot provide to ChatGPT, not only for legal reasons, but just due to the amount of code that would be required to provide enough context for the LLM to chew on. Even personal projects, where I'm learning new languages and libraries, I've found that the code that gets generated in most cases is incorrect at best, and won't compile at worst. So I have to go through and double-check all of its "work" anyway- just like I'd have to do if I had a junior engineer sidekick who didn't know how to run the compiler. I think for the work problems, if our company could train and self-host an LLM system on all of our internal code, it would be interesting to see if that could be used to assist building out new features and fixes.
- ukuina 3y ago
  This is the use-case that OpenAI on Azure is trying to solve. It supports finetuning, too. Not cheap, though. I think most companies will end up hosting an LLM on local code rather than using OpenAI on Azure, but for now, there is no equivalently-capable replacement.

by jstx1 3y ago

jabradoodle 3y ago

The parent is asking for a specific example use case.

hayksaakian 3y ago
I can't speak for OP, but for me I literally never use stack overflow any more, and I spend about 90% less time on Google
- styfle 3y ago
  Curious if that’s because AI provides better answers? It’s certainly not quicker answers, right?
- 8organicbits 3y ago
  Are you not fact checking chatgpt? I've seen wrong info, especially subtle things. It seemed reckless to use as-is.
  - blowski 3y ago
    Both ChatGPT and StackOverflow suffer from content becoming outdated. So some highly-upvoted answer on StackOverflow has been out of date since 2011, and now ChatGPT is trained on it. I see the future as writing test cases (perhaps also with ChatGPT), and separately using ChatGPT to write the implementation. Perhaps we will just give it a bunch of test cases and it will return code (or submit a PR) that passes those tests.
  - etimberg 3y ago
    For fun I tried asking chatgpt to create a simple using an opensource project I maintain. The generated answer was sort of correct but not correct enough to copy and paste. It missed including a plugin, used a version of the project that doesn't exist yet, and generated data that wasn't valid datetimes.
- hallway_monitor 3y ago
  Yep exactly. I guess I haven't hit the 25 messages in 3 hours limit, but whenever there's an API or library I'm not familiar with. I can get my exact example in about 10 seconds from ChatGPT 4
  - iudqnolq 3y ago
    Are those popular APIs? I've found Copilot useful when writing greenfield code, but very unhelpful generating code that uses APIs not popular enough to have significant coverage on StackOverflow. Even if I have examples of correct usage in the same file it still guesses plausible but wrong types. I haven't bought GPT 4 but I'm curious if it's much better at this.
    - lstamour 3y ago
      If you don't mention a library by name it is liable to make something up by picking a popular library in another language and converting the syntax to the language you asked for. If you ask for something impossible in a library it will also frequently make up functions or application settings. If you ask for something obscure but hard to do, it might reply that it's impossible but it is possible if you know how and teach it. I sort of compare prompt engineering to Googling - you sometimes have to search for exactly the right terms that you want to appear in the result in order to get the answer you're looking for. It's just that the flexibility of ChatGPT in writing a direct response sometimes means it will completely make up an answer. There's also a limitation that the web interface doesn't actually let you upload files and has a length limit for inputs. For Copilot, I'm looking forward to Copilot X: https://www.youtube.com/watch?v=3surPGP7_4o https://www.youtube.com/watch?v=3surPGP7_4o
      - iudqnolq 3y ago
        This was neither. I've forgotten the exact words I typed but it was something like this. Prompt: fn encode(value: Foo) { capnproto::serialize_packed:: serialize_message(value); } fn decode(input: &[u8]) { Expected: capnproto::serialize_packed:: deserialize_message(input); Generated capnproto::PackedMessageDeserializer::deserialize(input)
michaelbuckbee 3y ago
An eye opening example for me was that I was working with a Ruby/Rails class that was testing various IP configurations [1] and I was able to just copy and paste it into chatgpt and say "write some tests for this". It wasn't really anything I couldn't have written in a half hour or so but it was so much faster. The real kicker is that by default chatgpt wrote Rspec and I was able to say "rewrite that in minitest" and it worked. 1 - https://wafris.org/ip-lookup https://wafris.org/ip-lookup

nice_byte 3y ago

what kind of problems are you trying to solve that make gpt-4 so helpful to you?

cbm-vic-20 3y ago

I'm really trying to do the same, for both my work, and personal projects. But the type of answers I need for work (enterprise software, large codebase built over 20+ years) requires a ton of context that I simply cannot provide to ChatGPT, not only for legal reasons, but just due to the amount of code that would be required to provide enough context for the LLM to chew on. Even personal projects, where I'm learning new languages and libraries, I've found that the code that gets generated in most cases is incorrect at best, and won't compile at worst. So I have to go through and double-check all of its "work" anyway- just like I'd have to do if I had a junior engineer sidekick who didn't know how to run the compiler. I think for the work problems, if our company could train and self-host an LLM system on all of our internal code, it would be interesting to see if that could be used to assist building out new features and fixes.

ukuina 3y ago
This is the use-case that OpenAI on Azure is trying to solve. It supports finetuning, too. Not cheap, though. I think most companies will end up hosting an LLM on local code rather than using OpenAI on Azure, but for now, there is no equivalently-capable replacement.