5 ms·

When it comes to programming, I agree completely. The sweet spot for any use of LLMs is that you already know enough about the subject to verify the work - at l

When it comes to programming, I agree completely. The sweet spot for any use of LLMs is that you already know enough about the subject to verify the work - at least the output - and know enough how to describe in detail (ideally only salient details) what you want. Huge +1 to it helping me do things faster, do things that I wouldnt have otherwise done, or using it for throwaway, mostly inconsequential yet valuable programs. But another area I have found it extremely helpful in is exploring a new topic entirely, programming or otherwise. Telling it that I dont know what im talking about, don't necessarily need specifics, but here is what I want to talk about and want it to help me think through. Especially if you are that person who is willing to take what you hear and do more research or ask more question. The entrance to so many fields and subjects is just understanding the basic jargon, listening for the distinctions being made and understanding why, and knowing who the authorities are on the subject.

netcraft 3y ago
I'll add another thought here - what I really want many times is a custom LLM like GPT, but trained on a particular language or framework or topic. I would love to go to a website for a new language and be able to talk about its documentation and ask questions of an LLM to help me understand. Huge bonus points if it was trained on real world code examples of that language or framework and I could have it help me write a new program or function right there. More bonus points if its tied in with an online repl where it can help me right inline.
- crq-yml 3y ago
  Ease of retraining/refinement is something I'm really hoping for. There are an endless number projects to make a "cleaner, revised X", where the coding itself is rote and has already been done at some point, it's just shoved into slightly different semantics that will be a bit more optimal or secure or configurable. It's something that an LLM feels like it's "tip of the tongue" capable of, and in more trivial cases you really can tell GPT to "rewrite this from JS to Python" and it works. But it's limited by just interpolating what's in the training set, when what you want is "port all these standard libraries to my experimental language, and also make a build system for them".
j4yav 3y ago
And it's equally and inversely harmful to junior developers who keep prodding it until it generates an abomination they don't understand that manages to pass the build. People who are learning need help, but the kind of help that LLMs in copilot form provide aren't the right fit. It would be interesting to train a copilot model that is specifically intended to ask clarifying questions and be a partner in determining a solution, rather than doing its best to generate code for a vague or incorrectly specified question from a junior.
- makk 3y ago
  Have you tried prompting it to ask clarifying questions and be that partner? Perhaps no (extra) training required.
  - j4yav 3y ago
    In my opinion the junior developers are not equipped to guide their teacher. If they knew they were asking to turn an incorrect assumption into code in the first place, they already wouldn’t believe the confident hallucination they get in reply.
- CaptainFever 3y ago
  > And it's equally and inversely harmful to junior developers who keep prodding it until it generates an abomination they don't understand that manages to pass the build. This sounds like shotgun debugging.
  - j4yav 3y ago
    While on LSD in this case.
    - deleted 3y ago
      [deleted]
- ajmurmann 3y ago
  It might not be good for juniors, but, as Antirez points out, is great for adapting to new frameworks or APIs, especially with search results having become so much more noisy lately
itomato 3y ago
The One and Six Pagers I have it write based on loose criteria help me refine the criteria and in some cases, uncover methods that would not have been evident otherwise.
anileated 3y ago
Last month I tried to use LLMs for things I didn’t know and couldn’t easily find. Every time they were either subtly wrong or outright hallucinated premises which led me to waste time until I realized they were wrong. If not for the unwarranted confidence in incorrect responses, I could say they were at least not much worse than what I could piece together from what I knew. As it stands, they are OK filling in for a rubber duck and as autocomplete.
- _giorgio_ 3y ago
  Surely a bad LLM, not chatGPT 4
  - jay_kyburz 3y ago
    I can't tell if you are joking or not.
    - _giorgio_ 3y ago
      No
deleted 3y ago
[deleted]

netcraft 3y ago

I'll add another thought here - what I really want many times is a custom LLM like GPT, but trained on a particular language or framework or topic. I would love to go to a website for a new language and be able to talk about its documentation and ask questions of an LLM to help me understand. Huge bonus points if it was trained on real world code examples of that language or framework and I could have it help me write a new program or function right there. More bonus points if its tied in with an online repl where it can help me right inline.

crq-yml 3y ago
Ease of retraining/refinement is something I'm really hoping for. There are an endless number projects to make a "cleaner, revised X", where the coding itself is rote and has already been done at some point, it's just shoved into slightly different semantics that will be a bit more optimal or secure or configurable. It's something that an LLM feels like it's "tip of the tongue" capable of, and in more trivial cases you really can tell GPT to "rewrite this from JS to Python" and it works. But it's limited by just interpolating what's in the training set, when what you want is "port all these standard libraries to my experimental language, and also make a build system for them".

j4yav 3y ago

And it's equally and inversely harmful to junior developers who keep prodding it until it generates an abomination they don't understand that manages to pass the build. People who are learning need help, but the kind of help that LLMs in copilot form provide aren't the right fit. It would be interesting to train a copilot model that is specifically intended to ask clarifying questions and be a partner in determining a solution, rather than doing its best to generate code for a vague or incorrectly specified question from a junior.

makk 3y ago
Have you tried prompting it to ask clarifying questions and be that partner? Perhaps no (extra) training required.
- j4yav 3y ago
  In my opinion the junior developers are not equipped to guide their teacher. If they knew they were asking to turn an incorrect assumption into code in the first place, they already wouldn’t believe the confident hallucination they get in reply.
CaptainFever 3y ago
> And it's equally and inversely harmful to junior developers who keep prodding it until it generates an abomination they don't understand that manages to pass the build. This sounds like shotgun debugging.
- j4yav 3y ago
  While on LSD in this case.
  - deleted 3y ago
    [deleted]
ajmurmann 3y ago
It might not be good for juniors, but, as Antirez points out, is great for adapting to new frameworks or APIs, especially with search results having become so much more noisy lately

itomato 3y ago

The One and Six Pagers I have it write based on loose criteria help me refine the criteria and in some cases, uncover methods that would not have been evident otherwise.

anileated 3y ago

Last month I tried to use LLMs for things I didn’t know and couldn’t easily find. Every time they were either subtly wrong or outright hallucinated premises which led me to waste time until I realized they were wrong. If not for the unwarranted confidence in incorrect responses, I could say they were at least not much worse than what I could piece together from what I knew. As it stands, they are OK filling in for a rubber duck and as autocomplete.

_giorgio_ 3y ago
Surely a bad LLM, not chatGPT 4
- jay_kyburz 3y ago
  I can't tell if you are joking or not.
  - _giorgio_ 3y ago
    No

deleted 3y ago

[deleted]