4 ms·

From my experiments so far, I am unsure if chatgpt is effective at optimizing prompts, even though it thinks it is.

capableweb 3y ago
The workflow you get with this is that it helps you to think about what you really want answered with the prompt. For example, latest try had this initial prompt: > How can I create collisions in Bevy and Rust? After 4-5 messages back and forth, I ended up with: > What is the most efficient method to implement 2D collision detection for a large number of constant-radius circles randomly distributed in Bevy and Rust without using third-party tools, focusing on detection only, without any unique properties or attributes affecting the detection process? Which is much more explicit and targeted to what I initially wanted to do, but didn't write. It ended up helping me implement a Quadtree solution which I don't have any experience with before, but overall it went smoothly.
- charcircuit 3y ago
  How do you know that the prompt you got to is at least a local maximum? Would chancing "method" to "function" provide better results? Did you do any benchmarking?
  - capableweb 3y ago
    The aim is not any local maximum, is improving the prompt by getting help providing more context.
    - charcircuit 3y ago
      That's a pretty bad optimizer then.
ryanjshaw 3y ago
I've seen it help me once so far in a way I hadn't thought of: basically, the chat completion engine was reading a bug bounty report on my behalf, and believing generic, vague claims. ChatGPT-3.5 didn't "fall" for this, but 4 did consistently. After failing to improve performance manually, I eventually asked 4 to improve my prompt after explaining the problem and it basically added "Don't assume the existence of any examples". This makes perfect sense in hindsight, but I had been approaching things from the other direction - directing it to only consider explicit examples in the report, and for some reason that didn't work.

The workflow you get with this is that it helps you to think about what you really want answered with the prompt. For example, latest try had this initial prompt: > How can I create collisions in Bevy and Rust? After 4-5 messages back and forth, I ended up with: > What is the most efficient method to implement 2D collision detection for a large number of constant-radius circles randomly distributed in Bevy and Rust without using third-party tools, focusing on detection only, without any unique properties or attributes affecting the detection process? Which is much more explicit and targeted to what I initially wanted to do, but didn't write. It ended up helping me implement a Quadtree solution which I don't have any experience with before, but overall it went smoothly.

charcircuit 3y ago
How do you know that the prompt you got to is at least a local maximum? Would chancing "method" to "function" provide better results? Did you do any benchmarking?
- capableweb 3y ago
  The aim is not any local maximum, is improving the prompt by getting help providing more context.
  - charcircuit 3y ago
    That's a pretty bad optimizer then.

ryanjshaw 3y ago

I've seen it help me once so far in a way I hadn't thought of: basically, the chat completion engine was reading a bug bounty report on my behalf, and believing generic, vague claims. ChatGPT-3.5 didn't "fall" for this, but 4 did consistently. After failing to improve performance manually, I eventually asked 4 to improve my prompt after explaining the problem and it basically added "Don't assume the existence of any examples". This makes perfect sense in hindsight, but I had been approaching things from the other direction - directing it to only consider explicit examples in the report, and for some reason that didn't work.