3 ms·

I can’t find the link to the paper right now, but after reading about how LLMs perform better with task breakdowns, I vastly improved my integrations by having

I can’t find the link to the paper right now, but after reading about how LLMs perform better with task breakdowns, I vastly improved my integrations by having ChatGPT generate prompts that decompose a general task into a series of tasks based on a sample input and output. I haven’t needed to make a self-refining system (one or two rounds of task decomposition and refinement resulted in the expected result for all inputs), but I would assume this is fairly trivial and that AIs can do it better than humans. This is also an area where I expect OpenAI will continue to demolish the competition. The ability to recursively generate and process large prompts is truly nuts. I tried swapping in some of the “high-performing” LLama models and they all choked on anything more than a paragraph.

bravogamma 4y ago
Will you share examples for prompts that “vastly improved” your integrations?
- bugglebeetle 4y ago
  The prompts are just all general NLP stuff, but with the addition of a series of tasks, generated by ChatGPT. For example, you start with a prompt like “You are an AI assistant capable of classifying text into one of four categories, A, B, C, D” and then add something like “Assume as input X with expected result Y? How would you accomplish this? Break it down into a detailed series of tasks.” ChatGPT will then decompose the general task into the detailed steps that you can review and tweak as needed. Then, you just restructure the prompt like “ You are an AI assistant capable of classifying text into one of four categories, A, B, C, D using a series of tasks. For example, if provided input X, you would [series of tasks provided provided by ChatGPT] and return output Y. Now, classify the following text in the same manner: [new input]”
  - tough 4y ago
    I think you're talking about the ReAct (Reason -> Act) pattern https://til.simonwillison.net/llms/python-react-pattern https://til.simonwillison.net/llms/python-react-pattern
axiom92 4y ago
For those curious about self-refining systems: https://selfrefine.info/ https://selfrefine.info/ (our recent work).

bravogamma 4y ago

Will you share examples for prompts that “vastly improved” your integrations?

bugglebeetle 4y ago
The prompts are just all general NLP stuff, but with the addition of a series of tasks, generated by ChatGPT. For example, you start with a prompt like “You are an AI assistant capable of classifying text into one of four categories, A, B, C, D” and then add something like “Assume as input X with expected result Y? How would you accomplish this? Break it down into a detailed series of tasks.” ChatGPT will then decompose the general task into the detailed steps that you can review and tweak as needed. Then, you just restructure the prompt like “ You are an AI assistant capable of classifying text into one of four categories, A, B, C, D using a series of tasks. For example, if provided input X, you would [series of tasks provided provided by ChatGPT] and return output Y. Now, classify the following text in the same manner: [new input]”
- tough 4y ago
  I think you're talking about the ReAct (Reason -> Act) pattern https://til.simonwillison.net/llms/python-react-pattern https://til.simonwillison.net/llms/python-react-pattern

axiom92 4y ago

For those curious about self-refining systems: https://selfrefine.info/ https://selfrefine.info/ (our recent work).