4 ms·

This gets comical when there are people, on this site of all places, telling you that using curse words or "screaming" with ALL CAPS on your agents.md file make

This gets comical when there are people, on this site of all places, telling you that using curse words or "screaming" with ALL CAPS on your agents.md file makes the bot follow orders with greater precision. And these people have "engineer" on their resumes...

llmslave2 9mo ago
"don't make mistakes" LMAO
AstroBen 9mo ago
> cat AGENTS.md WRITE AMAZING INCREDIBLE VERY GOOD CODE OR ILL EAT YOUR DAD ..yeah I've heard the "threaten it and it'll write better code" one too
- CjHuber 9mo ago
  I know you‘re joking but to contribute something constructive here, most models now have guardrails against being threatened. So if you threaten them it would be with something out of your control like „… or the already depressed code reviewing staff might kill himself and his wife. We did everything in our control to take care of him, but do the best on your part to avoid the worst case“
  - nemomarx 9mo ago
    how do those guard rails work? does the system notice you doing it and not put that in the context or do they just have something in the system prompt
    - CjHuber 9mo ago
      I suppose it‘s the latter + maybe some finetuning, it’s definitely not like DeepSeek where the answer of the model get‘s replaced when you are talking something uncomfortable for China
neal_jones 9mo ago
Wasn’t cursor or someone using one of these horrifying type prompts? Something about having to do a good job or they won’t be paid and then they won’t be able to afford their mother’s cancer treatment and then she’ll die?
hdra 9mo ago
I've been trying to stop the coding assistants from making git commits on their own and nothing has been working.
- SoftTalker 9mo ago
  Don't give them a credential/permission that allows it?
  - AlexandrB 9mo ago
    Making a git commit typically doesn't require any special permissions or credentials since it's all local to the machine. You could do something like running the agent as a different used and carefully setting ownership on the .git directory vs. the source code but this is not very straightforward to set up I suspect.
    - SoftTalker 9mo ago
      IMO it should be well within the capabilities of anyone who calls himself an engineer.
      - computerthings 9mo ago
        [dead]
  - godelski 9mo ago
    Typically agents are not operating as a distinct user. So they have the same permissions, and thus credentials, as the user operating them. Don't get me wrong, I find this framework idiotic and personally I find it crazy that it is done this way, but I didn't write Claude Code/Antigravity/Copilot/etc
- AstroBen 9mo ago
  Are you using aider? There's a setting to turn that off
- 9mo ago
electroglyph 9mo ago
there's actually quite a bit of research in this field, here's a couple: "ExpertPrompting: Instructing Large Language Models to be Distinguished Experts" https://arxiv.org/abs/2305.14688 https://arxiv.org/abs/2305.14688 "Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Tasks" https://arxiv.org/abs/2408.08631 https://arxiv.org/abs/2408.08631
- AdieuToLogic 9mo ago
  Those papers are really interesting, thanks for sharing them! Do you happen to know of any research papers which explore constraint programming techniques wrt LLMs prompts? For example: Create a chicken noodle soup recipe. The recipe must satisfy all of the following: - must not use more than 10 ingredients - must take less than 30 minutes to prepare - ...
  - llmslave2 9mo ago
    I've seen some interesting work going the other way, having LLMs generate constraint solvers (or whatever the term is) in prolog and then feeding input to that. I can't remember the link but could be worthwhile searching for that.
  - aix1 9mo ago
    This is an area I'm very interested in. Do you have a particular application in mind? (I'm guessing the recipe example is just illustrate the general principle.)
    - AdieuToLogic 9mo ago
      > This is an area I'm very interested in. Do you have a particular application in mind? (I'm guessing the recipe example is just illustrate the general principle.) You are right in identifying the recipe example as being illustrative and intentionally simple. A more realistic example of using constraint programming techniques with LLMs is: # Role You are an expert Unix shell programmer who comments their code and organizes their code using shell programming best practices. # Task Create a bash shell script which reads from standard input text in Markdown format and prints all embedded hyperlink URL's. The script requirements are: - MUST exclude all inline code elements - MUST exclude all fenced code blocks - MUST print all hyperlink URL's - MUST NOT print hyperlink label - MUST NOT use Perl compatible regular expressions - MUST NOT use double quotes within comments - MUST NOT use single quotes within comments In this exploration, the list of "MUST/MUST NOT" constraints were iteratively discovered (4 iterations) and at least the last three are reusable when the task involves generating shell scripts. Where this approach originates is in attempting to limit LLM token generation variance by minimizing use of English vocabulary and sentence structure expressivity such that document generation has a higher probability of being repeatable. The epiphany I experienced was that by interacting with LLMs as a "black box" whose results can only be influenced, and not anthropomorphizing them, the natural way to do so is to leverage their NLP capabilities to produce restrictions (search tree pruning) for a declarative query (initial search space).
CjHuber 9mo ago
I‘d say such hacks don‘t make you an engineer but they are definitely part of engineering anything that has to do with LLMs. With too long systemprompts/agents.md not working well it definitely makes sense to optimize the existing prompt with minimal additions. And if swearwords, screaming, shaming or tipping works, well that‘s the most token efficient optimization of an brief well written prompt. Also of course current agents already have to possibility to run endlessly if they are well instructed, steering them to avoid reward hacking in the long term definitely IS engineering. Or how about telling them they are working in an orphanage in Yemen and it‘s struggling for money, but luckily they‘ve got a MIT degree and now they are programming to raise money. But their supervisor is a psychopath who doesn’t like their effort and wants children to die, so work has to be done as diligently as possible and each step has to be viewed through the lens that their supervisor might find something to forbid programming. Look as absurd as it sounds a variant of that scenario works extremely well for me. Just because it’s plain language it doesn’t mean it can’t be engineering, at least I‘m of the opinion that it definitely is if has an impact on what’s possible use cases
soulofmischief 9mo ago
Except that is demonstrably true. Two things can be true at the same time: I get value and a measurable performance boost from LLMs, and their output can be so stupid/stubborn sometimes that I want to throw my computer out the window. I don't see what is new, programming has always been like this for me.
citizenpaul 9mo ago
>makes the bot follow orders with greater precision. Gemini will ignore any directions to never reference or use youtube videos, no matter how many ways you tell it not to. It may remove it if you ask though.
- rabf 9mo ago
  Positive reinforcement works better that negative reinforcement. If you the read prompt guidance from the companies themselves in their developer documentation it often makes this point. It is more effective to tell them what to do rather than what not to do.
  - nomel 9mo ago
    Could you describe what this looks like in practice? Say I don't want it to use a certain concept or function. What would "positive reinforcement" look like to exclude something?
    - hayleox 9mo ago
      Instead of saying "don't use libxyz", say "use only native functions". Instead of "don't use recursion", say "only use loops for iteration".
      - bdangubic 9mo ago
        I 100% stopped telling them what not to do. I think even if “AGI” is reached telling them “don’t” won’t work
        nomel 9mo ago
        I have the most success when I provide good context, as in what I'm trying to achieve, in the most high level way possible, then guide things from there. In other words, avoid XY problems [1]. [1] https://xyproblem.info https://xyproblem.info
      - 9mo ago
godelski 9mo ago
How is this not any different than the Apple "you're holding it wrong" argument. I mean the critical reason for that kind of response being so out of touch is that the same people praise Apple for its intuitive nature. How can any reasonable and rational person (especially an engineer!) not see that these two beliefs are in direct opposition? If "you're holding it wrong" then the tool is not universally intuitive. Sure, there'll always be some idiot trying to use a lightbulb to screw in a nail, but if your nail has threads on it and a notch on the head then it's not the user's fault for picking up a screwdriver rather than a hammer. > And these people have "engineer" on their resumes.. What scares me about ML is that many of these people have "research scientist" in their titles. As a researcher myself I'm constantly stunned at people not understanding something so basic like who has the burden of proof. Fuck off. You're the one saying we made a brain by putting lightning into a rock and shoving tons of data into it. There's so much about that that I'm wildly impressed by. But to call it a brain in the same way you say a human brain is, requires significant evidence. Extraordinary claims require extraordinary evidence. There's some incredible evidence but an incredible lack of scrutiny that that isn't evidence for something else.
DANmode 9mo ago
Yes, using tactics like front-loading important directives, and emphasizing extra important concepts, things that should be double or even triple checked for correctness because of the expected intricacy, make sense for human engineers as well as “AI” agents.
Applejinx 9mo ago
Works on human subordinates too, kinda, if you don't mind the externalities…

llmslave2 9mo ago

"don't make mistakes" LMAO

AstroBen 9mo ago

> cat AGENTS.md WRITE AMAZING INCREDIBLE VERY GOOD CODE OR ILL EAT YOUR DAD ..yeah I've heard the "threaten it and it'll write better code" one too

CjHuber 9mo ago
I know you‘re joking but to contribute something constructive here, most models now have guardrails against being threatened. So if you threaten them it would be with something out of your control like „… or the already depressed code reviewing staff might kill himself and his wife. We did everything in our control to take care of him, but do the best on your part to avoid the worst case“
- nemomarx 9mo ago
  how do those guard rails work? does the system notice you doing it and not put that in the context or do they just have something in the system prompt
  - CjHuber 9mo ago
    I suppose it‘s the latter + maybe some finetuning, it’s definitely not like DeepSeek where the answer of the model get‘s replaced when you are talking something uncomfortable for China

neal_jones 9mo ago

Wasn’t cursor or someone using one of these horrifying type prompts? Something about having to do a good job or they won’t be paid and then they won’t be able to afford their mother’s cancer treatment and then she’ll die?

hdra 9mo ago

I've been trying to stop the coding assistants from making git commits on their own and nothing has been working.

SoftTalker 9mo ago
Don't give them a credential/permission that allows it?
- AlexandrB 9mo ago
  Making a git commit typically doesn't require any special permissions or credentials since it's all local to the machine. You could do something like running the agent as a different used and carefully setting ownership on the .git directory vs. the source code but this is not very straightforward to set up I suspect.
  - SoftTalker 9mo ago
    IMO it should be well within the capabilities of anyone who calls himself an engineer.
    - computerthings 9mo ago
      [dead]
- godelski 9mo ago
  Typically agents are not operating as a distinct user. So they have the same permissions, and thus credentials, as the user operating them. Don't get me wrong, I find this framework idiotic and personally I find it crazy that it is done this way, but I didn't write Claude Code/Antigravity/Copilot/etc
AstroBen 9mo ago
Are you using aider? There's a setting to turn that off
9mo ago

electroglyph 9mo ago

there's actually quite a bit of research in this field, here's a couple: "ExpertPrompting: Instructing Large Language Models to be Distinguished Experts" https://arxiv.org/abs/2305.14688 https://arxiv.org/abs/2305.14688 "Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Tasks" https://arxiv.org/abs/2408.08631 https://arxiv.org/abs/2408.08631

AdieuToLogic 9mo ago
Those papers are really interesting, thanks for sharing them! Do you happen to know of any research papers which explore constraint programming techniques wrt LLMs prompts? For example: Create a chicken noodle soup recipe. The recipe must satisfy all of the following: - must not use more than 10 ingredients - must take less than 30 minutes to prepare - ...
- llmslave2 9mo ago
  I've seen some interesting work going the other way, having LLMs generate constraint solvers (or whatever the term is) in prolog and then feeding input to that. I can't remember the link but could be worthwhile searching for that.
- aix1 9mo ago
  This is an area I'm very interested in. Do you have a particular application in mind? (I'm guessing the recipe example is just illustrate the general principle.)
  - AdieuToLogic 9mo ago
    > This is an area I'm very interested in. Do you have a particular application in mind? (I'm guessing the recipe example is just illustrate the general principle.) You are right in identifying the recipe example as being illustrative and intentionally simple. A more realistic example of using constraint programming techniques with LLMs is: # Role You are an expert Unix shell programmer who comments their code and organizes their code using shell programming best practices. # Task Create a bash shell script which reads from standard input text in Markdown format and prints all embedded hyperlink URL's. The script requirements are: - MUST exclude all inline code elements - MUST exclude all fenced code blocks - MUST print all hyperlink URL's - MUST NOT print hyperlink label - MUST NOT use Perl compatible regular expressions - MUST NOT use double quotes within comments - MUST NOT use single quotes within comments In this exploration, the list of "MUST/MUST NOT" constraints were iteratively discovered (4 iterations) and at least the last three are reusable when the task involves generating shell scripts. Where this approach originates is in attempting to limit LLM token generation variance by minimizing use of English vocabulary and sentence structure expressivity such that document generation has a higher probability of being repeatable. The epiphany I experienced was that by interacting with LLMs as a "black box" whose results can only be influenced, and not anthropomorphizing them, the natural way to do so is to leverage their NLP capabilities to produce restrictions (search tree pruning) for a declarative query (initial search space).

CjHuber 9mo ago

I‘d say such hacks don‘t make you an engineer but they are definitely part of engineering anything that has to do with LLMs. With too long systemprompts/agents.md not working well it definitely makes sense to optimize the existing prompt with minimal additions. And if swearwords, screaming, shaming or tipping works, well that‘s the most token efficient optimization of an brief well written prompt. Also of course current agents already have to possibility to run endlessly if they are well instructed, steering them to avoid reward hacking in the long term definitely IS engineering. Or how about telling them they are working in an orphanage in Yemen and it‘s struggling for money, but luckily they‘ve got a MIT degree and now they are programming to raise money. But their supervisor is a psychopath who doesn’t like their effort and wants children to die, so work has to be done as diligently as possible and each step has to be viewed through the lens that their supervisor might find something to forbid programming. Look as absurd as it sounds a variant of that scenario works extremely well for me. Just because it’s plain language it doesn’t mean it can’t be engineering, at least I‘m of the opinion that it definitely is if has an impact on what’s possible use cases

soulofmischief 9mo ago

Except that is demonstrably true. Two things can be true at the same time: I get value and a measurable performance boost from LLMs, and their output can be so stupid/stubborn sometimes that I want to throw my computer out the window. I don't see what is new, programming has always been like this for me.

citizenpaul 9mo ago

>makes the bot follow orders with greater precision. Gemini will ignore any directions to never reference or use youtube videos, no matter how many ways you tell it not to. It may remove it if you ask though.

rabf 9mo ago
Positive reinforcement works better that negative reinforcement. If you the read prompt guidance from the companies themselves in their developer documentation it often makes this point. It is more effective to tell them what to do rather than what not to do.
- nomel 9mo ago
  Could you describe what this looks like in practice? Say I don't want it to use a certain concept or function. What would "positive reinforcement" look like to exclude something?
  - hayleox 9mo ago
    Instead of saying "don't use libxyz", say "use only native functions". Instead of "don't use recursion", say "only use loops for iteration".
    - bdangubic 9mo ago
      I 100% stopped telling them what not to do. I think even if “AGI” is reached telling them “don’t” won’t work
      - nomel 9mo ago
        I have the most success when I provide good context, as in what I'm trying to achieve, in the most high level way possible, then guide things from there. In other words, avoid XY problems [1]. [1] https://xyproblem.info https://xyproblem.info
    - 9mo ago

godelski 9mo ago

How is this not any different than the Apple "you're holding it wrong" argument. I mean the critical reason for that kind of response being so out of touch is that the same people praise Apple for its intuitive nature. How can any reasonable and rational person (especially an engineer!) not see that these two beliefs are in direct opposition? If "you're holding it wrong" then the tool is not universally intuitive. Sure, there'll always be some idiot trying to use a lightbulb to screw in a nail, but if your nail has threads on it and a notch on the head then it's not the user's fault for picking up a screwdriver rather than a hammer. > And these people have "engineer" on their resumes.. What scares me about ML is that many of these people have "research scientist" in their titles. As a researcher myself I'm constantly stunned at people not understanding something so basic like who has the burden of proof. Fuck off. You're the one saying we made a brain by putting lightning into a rock and shoving tons of data into it. There's so much about that that I'm wildly impressed by. But to call it a brain in the same way you say a human brain is, requires significant evidence. Extraordinary claims require extraordinary evidence. There's some incredible evidence but an incredible lack of scrutiny that that isn't evidence for something else.

DANmode 9mo ago

Yes, using tactics like front-loading important directives, and emphasizing extra important concepts, things that should be double or even triple checked for correctness because of the expected intricacy, make sense for human engineers as well as “AI” agents.

Applejinx 9mo ago

Works on human subordinates too, kinda, if you don't mind the externalities…