4 ms·

Unless they were using a model too stupid for this operation, the fundamental problem is usually solvable via prompting. Usually the issue is that the models h

Unless they were using a model too stupid for this operation, the fundamental problem is usually solvable via prompting. Usually the issue is that the models have a bias for action, so you need to give it an accetpable action when there isn't a good comment. Some other output/determination. I've seen this in many other similar applications.

deleted 2y ago
[deleted]
marviel 2y ago
+1 for "No Comment/Action Required" responses reducing trigger-happiness
keepingscore 2y ago
This isn't my post but I work with llms everyday and I've seen this kind of instruction ignoring behavior on sonnet when the context window starts getting close to the edge.
- XenophileJKO 2y ago
  I don't know, in practice there are so many potential causes that you have to look case by case in situations like that. I don't have a ton of experience with the raw Claude model specifically, but would anticipate you'll have the same problem classes. Usually it comes down to one of the following: - ambiguity and semantics (I once had a significant behavior difference between "suggest" and "recommend", i.e. a model can suggest without recommending.) - conflicting instructions - data/instruction bleeding (delimiters help, but if the span is too long it can loose track of what is data and what is instructions.) - action bias (If the task is to find code comments for example, even if you tell it not to, it will have a bias to do it as you defined the task that way.) - exceeding attention capacity (having to pay attention to too much or having too many instructions. This is where structures output or chain of thought type approaches help. They help focus attention on each step of the process and the related rules.) I feel like these are the ones you encounter the most.
  - dakshgupta 2y ago
    One word changes impacting output is interesting but also quite frustrating. Especially because the patterns don’t translate across models.
    - gopher_space 2y ago
      The "write like the people who wrote the info you want" pattern absolutely translates across models.
      - sdesol 2y ago
        Yes and no. I've found the order in which you give instructions matters for some models as well. With LLMs, you really need to treat them like black boxes and you cannot assume one prompt will work for all. It is honestly, in my experience, a lot of trial and error.
- dakshgupta 2y ago
  This might be what we experienced. We regularly have context reach 30k+ tokens.
NBJack 2y ago
Prompting only works if the training corpus used for the LLM has critical mass on the concept. I agree that the model should have alternatives (i.e. preface the comment with the tag 'Nit:'), but bear in mind code review nit-picking isn't exactly a well-researched area of study. Also to your point, given this was also likely trained predominantly on code, there's probably a lack of comment sentiment in the corpus.
nitwit005 2y ago
If even multiple software developers cannot make a working prompt, the model just isn't good enough for what they want it to do.

deleted 2y ago

[deleted]

marviel 2y ago

+1 for "No Comment/Action Required" responses reducing trigger-happiness

keepingscore 2y ago

This isn't my post but I work with llms everyday and I've seen this kind of instruction ignoring behavior on sonnet when the context window starts getting close to the edge.

XenophileJKO 2y ago
I don't know, in practice there are so many potential causes that you have to look case by case in situations like that. I don't have a ton of experience with the raw Claude model specifically, but would anticipate you'll have the same problem classes. Usually it comes down to one of the following: - ambiguity and semantics (I once had a significant behavior difference between "suggest" and "recommend", i.e. a model can suggest without recommending.) - conflicting instructions - data/instruction bleeding (delimiters help, but if the span is too long it can loose track of what is data and what is instructions.) - action bias (If the task is to find code comments for example, even if you tell it not to, it will have a bias to do it as you defined the task that way.) - exceeding attention capacity (having to pay attention to too much or having too many instructions. This is where structures output or chain of thought type approaches help. They help focus attention on each step of the process and the related rules.) I feel like these are the ones you encounter the most.
- dakshgupta 2y ago
  One word changes impacting output is interesting but also quite frustrating. Especially because the patterns don’t translate across models.
  - gopher_space 2y ago
    The "write like the people who wrote the info you want" pattern absolutely translates across models.
    - sdesol 2y ago
      Yes and no. I've found the order in which you give instructions matters for some models as well. With LLMs, you really need to treat them like black boxes and you cannot assume one prompt will work for all. It is honestly, in my experience, a lot of trial and error.
dakshgupta 2y ago
This might be what we experienced. We regularly have context reach 30k+ tokens.

NBJack 2y ago

Prompting only works if the training corpus used for the LLM has critical mass on the concept. I agree that the model should have alternatives (i.e. preface the comment with the tag 'Nit:'), but bear in mind code review nit-picking isn't exactly a well-researched area of study. Also to your point, given this was also likely trained predominantly on code, there's probably a lack of comment sentiment in the corpus.

nitwit005 2y ago

If even multiple software developers cannot make a working prompt, the model just isn't good enough for what they want it to do.