4 ms·

I've really struggled every time I've pulled out any LLM for programming besides using Copilot for generating tests. Maybe I've been using it for the wrong thi

I've really struggled every time I've pulled out any LLM for programming besides using Copilot for generating tests. Maybe I've been using it for the wrong things—it certainly never helps unblock me when I'm stuck like it sounds like it does for some (I suspect it's because when I get stuck it's deep in undocumented rabbit holes), but it sounds like it might be decent at large-scale rote refactoring? Aside from specialized editors, how do people use it for things like that?

rvnx 2y ago
At least from my experience: You take Claude, you create a new Project, in your Project you explain the context of what you are doing and what you are programming (you have to explain it only once!). If you have specific technical documentation (e.g. rare programming language, your own framework, etc), you can put it there in the project. Then you create a conversation, and copy-paste the source-code for your file, and ask for your refactoring or improvement. If you are lazy just say: "give me the full code" and then "continue the code" few times in a row and you're done :)
- munksbeer 2y ago
  > in your Project you explain the context of what you are doing and what you are programming (you have to explain it only once!). When you say this, you mean typing out some text somewhere? Where do you do this? In a giant comment? In which file?
  - rvnx 2y ago
    In "Projects" -> "Create new project" -> "What are you trying to achieve?"
danielbln 2y ago
Provide context to the model. The code you're working on, what it's for, where you're stuckhat you've tried, etc. Pretend it's a colleague that should help you out and onboard it to your problem, then have a conversation with it as of your are rubber ducking your colleague. Don't ask short one-off questions and expect it to work (it might just, depending on what you ask, but probably not if you're deep on some proprietary code base with no traces in the LLMs pretraining).
- lolinder 2y ago
  I've definitely tried that and it doesn't work for the problems I've tried. Claude's answers for me always have all the hallmarks of an LLM response: extremely confident, filled with misunderstandings of even widely used APIs, and often requiring active correction on so many details that I'm not convinced it wouldn't have been faster to just search for a solution by hand. It feels like pair programming with a junior engineer, but without the benefit of helping train someone. I'm trying to figure out if I'm using it wrong or using it on the wrong types of problems. How do people with 10+ years of experience use it effectively?
  - danielbln 2y ago
    I think the only way to answer that is if you can share an example of a conversation you had with it, where it broke down as you described.
  - icholy 2y ago
    It's good for writing that prototype you're supposed to throw away. It's often easy to see the correct solution after seeing the wrong one.
  - klyrs 2y ago
    I'm a mathematician and the problems I work on tend to be quite novel (leetcode feel but with real-world applications). I find LLMs to be utterly useless at such tasks; "pair programming a junior, but without the benefit" is an excellent summary of my experience as well.
  - valval 2y ago
    For what I’m working on, I can also use the wrong approaches. Going through my fail often fail fast feedback loop is a lot more efficient with LLMs. Like A LOT more. Then when I have a bunch of wrong answers, I can give those as context as well to the model and make it avoid those pitfalls. At that point my constraints for the problem are so rigorous that the LLMs lands at the correct solution and frankly writes out the code 100x faster than I would. And I’m an advanced vim user who types at 155 wpm.

rvnx 2y ago

At least from my experience: You take Claude, you create a new Project, in your Project you explain the context of what you are doing and what you are programming (you have to explain it only once!). If you have specific technical documentation (e.g. rare programming language, your own framework, etc), you can put it there in the project. Then you create a conversation, and copy-paste the source-code for your file, and ask for your refactoring or improvement. If you are lazy just say: "give me the full code" and then "continue the code" few times in a row and you're done :)

munksbeer 2y ago
> in your Project you explain the context of what you are doing and what you are programming (you have to explain it only once!). When you say this, you mean typing out some text somewhere? Where do you do this? In a giant comment? In which file?
- rvnx 2y ago
  In "Projects" -> "Create new project" -> "What are you trying to achieve?"

danielbln 2y ago

Provide context to the model. The code you're working on, what it's for, where you're stuckhat you've tried, etc. Pretend it's a colleague that should help you out and onboard it to your problem, then have a conversation with it as of your are rubber ducking your colleague. Don't ask short one-off questions and expect it to work (it might just, depending on what you ask, but probably not if you're deep on some proprietary code base with no traces in the LLMs pretraining).

lolinder 2y ago
I've definitely tried that and it doesn't work for the problems I've tried. Claude's answers for me always have all the hallmarks of an LLM response: extremely confident, filled with misunderstandings of even widely used APIs, and often requiring active correction on so many details that I'm not convinced it wouldn't have been faster to just search for a solution by hand. It feels like pair programming with a junior engineer, but without the benefit of helping train someone. I'm trying to figure out if I'm using it wrong or using it on the wrong types of problems. How do people with 10+ years of experience use it effectively?
- danielbln 2y ago
  I think the only way to answer that is if you can share an example of a conversation you had with it, where it broke down as you described.
- icholy 2y ago
  It's good for writing that prototype you're supposed to throw away. It's often easy to see the correct solution after seeing the wrong one.
- klyrs 2y ago
  I'm a mathematician and the problems I work on tend to be quite novel (leetcode feel but with real-world applications). I find LLMs to be utterly useless at such tasks; "pair programming a junior, but without the benefit" is an excellent summary of my experience as well.
- valval 2y ago
  For what I’m working on, I can also use the wrong approaches. Going through my fail often fail fast feedback loop is a lot more efficient with LLMs. Like A LOT more. Then when I have a bunch of wrong answers, I can give those as context as well to the model and make it avoid those pitfalls. At that point my constraints for the problem are so rigorous that the LLMs lands at the correct solution and frankly writes out the code 100x faster than I would. And I’m an advanced vim user who types at 155 wpm.