4 ms·

This is an honest question. Did you try pasting your PR and the ChatGPT feedback into Claude and asking it for an analysis of the code and feedback?

verdverm 1y ago
Careful with this idea, I had someone take a thread we were engaged in and feed it to an LLM, asking it to confirm his feelings about the conversation, only to post it back to the group thread. It was used to attack me personally in a public space. Fortunately 1. The person was transparent about it, even posting a link to the chat session 2. They had to follow on prompt to really engage the sycophancy 3. The forum admins stepped in to speak to this individual even before I was aware of it I actually did what you suggested, fed everything back into another LLM, but did so with various prompts to test things out. The responses where... interesting, the positive prompt did return something quite good. A (paraphrased) quote from it "LLMs are a powerful rhetorical tool. Bringing one to a online discussion is like bringing a gun to a knife fight." That being said, how you prompt will get you wildly different responses from the same (other) inputs. I was able to get it to sycophant my (not actually) hurt feelings.
pavel_lishin 1y ago
Does that particularly matter in the context of this post? Either way, it sounds like OP was handed homework by the responder, and farming that out to yet another LLM seems kind of pointless, when OP could just ask the LLM for its opinion directly.
- uberman 1y ago
  While LLM code feedback might be wordy and dubious, I have personally found that asking Claude to review a PR and related feedback to provide some value. From my perspective anyways, Claude seems able to cut through the BS and say if a recommendation is worth the squeeze or in what contexts the feedback has merit or is just pedantic. Of course, your mileage my vary as they say.
  - pavel_lishin 1y ago
    Sure. But again, that's not what OP's post is about.

verdverm 1y ago

Careful with this idea, I had someone take a thread we were engaged in and feed it to an LLM, asking it to confirm his feelings about the conversation, only to post it back to the group thread. It was used to attack me personally in a public space. Fortunately 1. The person was transparent about it, even posting a link to the chat session 2. They had to follow on prompt to really engage the sycophancy 3. The forum admins stepped in to speak to this individual even before I was aware of it I actually did what you suggested, fed everything back into another LLM, but did so with various prompts to test things out. The responses where... interesting, the positive prompt did return something quite good. A (paraphrased) quote from it "LLMs are a powerful rhetorical tool. Bringing one to a online discussion is like bringing a gun to a knife fight." That being said, how you prompt will get you wildly different responses from the same (other) inputs. I was able to get it to sycophant my (not actually) hurt feelings.

pavel_lishin 1y ago

Does that particularly matter in the context of this post? Either way, it sounds like OP was handed homework by the responder, and farming that out to yet another LLM seems kind of pointless, when OP could just ask the LLM for its opinion directly.

uberman 1y ago
While LLM code feedback might be wordy and dubious, I have personally found that asking Claude to review a PR and related feedback to provide some value. From my perspective anyways, Claude seems able to cut through the BS and say if a recommendation is worth the squeeze or in what contexts the feedback has merit or is just pedantic. Of course, your mileage my vary as they say.
- pavel_lishin 1y ago
  Sure. But again, that's not what OP's post is about.