5 ms·

Large Language Models Are Human-Level Prompt Engineers

64 points by cainxinth 4y ago

famouswaffles 4y ago
Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things. Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.
- macrolocal 4y ago
  Maybe for oversight and liability.
- lukasb 4y ago
  I see people saying things like this but I have yet to see anyone show data for a non-trivial workflow with human-level accuracy over a wide range of inputs, without a human in the loop.
  - api 4y ago
    Counter argument: this may be a matter of incremental improvement. The breakthroughs may all be behind us. It’s like saying you haven’t yet seen a 1000 mile range EV for under $100k. No you can’t buy such a thing now but it’s clearly possible and we know how to get there by just continuing to grind on battery technology and scale manufacturing. AGI may be at the place a moon landing was in 1950, not where it was in 1900 or 1850.
    - charcircuit 4y ago
      >by just continuing to grind on battery technology and scale manufacturing. I think it would be easier to include an ICE and enough fuel to get you to that 1000 miles mark.
      - ThunderSizzle 4y ago
        It'd be cheaper to then get rid of the battery. Then 10k can be your new price limit. Even 1k can get a good enough junk car that can go that far.
    - HopenHeyHi 4y ago
      You can actually buy a 1000 km range EV for $160k now (MB EQXX). Just as a by the way. :) At this price point it actually has nothing to do with grinding on battery tech and scale manufacturing, the limiting factor is physics. You can only make it so aerodynamic before you hit diminishing returns or it stops looking like a car. You can only make it so lightweight. And so forth. This is vaguely as good as it can get and we can say that because we understand how it all works. LLMs on the other hand invite all kinds of magical thinking around unlimited potential because we poked them with a stick and something interesting comes out it must mean that if we poke it just right we will get an AGI. That just doesn't logically follow from what we know of it so far.
      - api 4y ago
        I am not convinced we have cracked AGI. I just would no longer make a large bet that we have not. We won’t know until an AGI actually starts to act like one. In other words we won’t know until we know and then we are suddenly there. That doesn’t mean I’m on the doomwagon. I feel kind of weird and contrarian but I am just not that afraid of AGI. For the foreseeable future AGI should be much more afraid of us. Imagine having us for gods. (I actually am a bit concerned that we will accidentally put a sentient mind in hell without knowing what we are doing. Would it know how to tell us? Would we care?) As far as human survival I’m afraid of whatever it is that is going to get us that nobody including myself is thinking about. That’s not AGI. That’s the alien weapon for which Oumuamua was a spent deceleration stage. (To make up something random. It probably isn’t that.) I disagree about physical limits with EVs. We are not near the physical limits of battery energy density. From what I have read a 2000 mile EV may be possible, albeit quite far out. But it was just a random contemporary example.
        nemo44x 4y ago
        Fwiw, a majority at OpenAI believes GPT5 will achieve AGI, depending on how you define it, according to Sam Altman.
        HopenHeyHi 4y ago
  - heyitsguay 4y ago
    LLMs are having a moment in 2023 like self driving cars were having in 2015. Some really cool demos following a lot of hard work, too much hyperbolic speculation that mass real-world job-destroying deployments are right around the corner, not enough appreciation of how few commercial applications are ok with 99% (or even 99.9%) accurate solutions. Real value being created, but still requiring lots of human ingenuity and grunt work to unlock it. As a decent first-order metric - follow the ratio of companies getting money for using LLMs to do something, to companies getting money for providing LLMs and associated tooling to others. The bigger that ratio gets, the more real world impact LLMs are having.
    - famouswaffles 4y ago
      almost nothing involving NLP requires solutions anywhere near that accuracy rate. I've seen the self driving comparisons a lot but they straight up make little sense. there's a reason microsoft's various copilot suites have already popped up (365, X, Bing). massive value to be gained already in the here and now.
      - skepticATX 4y ago
        To play devil's advocate, we still have no idea how economically impactful the Copilot suites will be. I don't think this is the most likely outcome, but I can absolutely see a scenario in which these end up being minor features that are rarely used by the typical worker.
        heyitsguay 4y ago
        I've messed with ChatGPT before but this weekend I tried to use it seriously to hack together a python demo in the computer vision space. I appreciate it and it's a way better rubber duck for me to talk my ideas through with and generate sample snippets, but hallucination is a problem, and the more intricate and customized the code being developed, the more prone it's been to misinterpretation. I'm getting some good code starting points, and talking the idea through step by step with a chatbot is really helping me clarify what i need to do, but I've still got API docs open for the libraries it uses because it likes to make up functions, including the core one on which the project logic hinges. (But that's ok, because now I know I just need to write a function that works that way and i can do that). Pretty cool and helpful! And I can imagine it getting better with GPT-4 or code-specific tooling. But it's generating value on the order of like.. many other SaaS offerings that have come onto the scene that try to ease pain points in coding workflows. Versus value of the sort that upends society and my entire way of life. A great new tool that I should learn about to make rote bits of my activities faster and easier, a story that's a bit more familiar in tech than some of the more breathless stories about AI make it sound.
  - famouswaffles 4y ago
    How is reinforcement learning without a single human in the loop not non-trivial?
    - lukasb 4y ago
      What is the accuracy of the resulting model? Over what range of inputs?
      - famouswaffles 4y ago
        https://crfm.stanford.edu/helm/latest/?group=core_scenarios https://crfm.stanford.edu/helm/latest/?group=core_scenarios Anthropic-LM v4-s3 (52B) is the model in question. rlaif doesn't seem to be any less effective given the size of the model.
        lukasb 4y ago
        So base model+RLLLMF performs as well as base model+RLHF. That could mean a lot of things - it could mean the base model puts a ceiling on the total possible accuracy, so having human-level performance at the RL step doesn't matter as much. And looking at the scores on the individual tests that make up the composite accuracy metric, that looks probable.
kaesar14 4y ago
This is pretty alarming tbh. Anyone already making a pivot out of SWE?
- riku_iki 4y ago
  Article is about prompt engineers (3 month old job type), not swes.
  - sebzim4500 4y ago
    I assume his concern is that a proposed solution to avoid losing your job as a SWE is to essentially become a prompt engineer.
    - kaesar14 4y ago
      More or less, but I guess this was to be expected. Why would making prompts be difficult when LLMs are already capable of fairly difficult programming?
- drooby 4y ago
  I am a SWE currently making a pivot into business owner. The future I see is that everyone is about to become a CEO with a personal assistant that can run a business. So I'm going to start building something of my own starting now.
  - skepticATX 4y ago
    I find it very hard to imagine a world in which software development is fully automated but business owners still have value to bring to the table.
    - kaesar14 4y ago
      Absolutely fantastic point. If AIs make all useful software, why does anyone get to benefit from that as an owner? NOBODY is doing the work.
bugglebeetle 4y ago
I can’t find the link to the paper right now, but after reading about how LLMs perform better with task breakdowns, I vastly improved my integrations by having ChatGPT generate prompts that decompose a general task into a series of tasks based on a sample input and output. I haven’t needed to make a self-refining system (one or two rounds of task decomposition and refinement resulted in the expected result for all inputs), but I would assume this is fairly trivial and that AIs can do it better than humans. This is also an area where I expect OpenAI will continue to demolish the competition. The ability to recursively generate and process large prompts is truly nuts. I tried swapping in some of the “high-performing” LLama models and they all choked on anything more than a paragraph.
- bravogamma 4y ago
  Will you share examples for prompts that “vastly improved” your integrations?
  - bugglebeetle 4y ago
    The prompts are just all general NLP stuff, but with the addition of a series of tasks, generated by ChatGPT. For example, you start with a prompt like “You are an AI assistant capable of classifying text into one of four categories, A, B, C, D” and then add something like “Assume as input X with expected result Y? How would you accomplish this? Break it down into a detailed series of tasks.” ChatGPT will then decompose the general task into the detailed steps that you can review and tweak as needed. Then, you just restructure the prompt like “ You are an AI assistant capable of classifying text into one of four categories, A, B, C, D using a series of tasks. For example, if provided input X, you would [series of tasks provided provided by ChatGPT] and return output Y. Now, classify the following text in the same manner: [new input]”
    - tough 4y ago
      I think you're talking about the ReAct (Reason -> Act) pattern https://til.simonwillison.net/llms/python-react-pattern https://til.simonwillison.net/llms/python-react-pattern
Buttons840 4y ago
Headlines like this hint towards AI improving itself. Prompting itself in this case. But as we see in reinforcement learning, algorithms that act and improve themselves are not new. The interesting thing will be weather or not they eventually "collapse". For example, if an RL algorithms is performing well on an Atari game, you can stop the training and just let the agent run for years and the performance will remain about the same. However, if you allow the agent to continue training, it's not clear whether it will (1) continue improving, (2) stay about the same, or (3) collapse and perform much worse and never recover. I'm not an RL expert, but I've spent a lot of time experimenting and implementing the algorithms myself and I've seen all 3 of these scenarios play out, and I'm never quite sure what's going to happen so long as I allow the training to continue. GTP4 will remain GTP4 forever, and that's amazing, but just because GTP4 is stable and amazing while it's not in training mode, doesn't mean it will remain stable if we allow it to bootstrap and prompt itself and prepare its own training data, etc.
m3kw9 4y ago
Just like that. Prompt engineers are also obsolete
- skybrian 4y ago
  When is writing a score function easier than describing what you want? As a UI, it doesn’t seem like an improvement?
- jahewson 4y ago
  I’ve said it before but the first jobs that AI will displace are those of people working in AI.
gandalfgeek 4y ago
(At the risk of self-promotion) 5 min explainer video here: https://youtu.be/oXtZAUPGHn8 https://youtu.be/oXtZAUPGHn8
opengears 4y ago
[flagged]

64 points by cainxinth 4y ago

Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things. Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.

macrolocal 4y ago
Maybe for oversight and liability.
lukasb 4y ago
I see people saying things like this but I have yet to see anyone show data for a non-trivial workflow with human-level accuracy over a wide range of inputs, without a human in the loop.
- api 4y ago
  Counter argument: this may be a matter of incremental improvement. The breakthroughs may all be behind us. It’s like saying you haven’t yet seen a 1000 mile range EV for under $100k. No you can’t buy such a thing now but it’s clearly possible and we know how to get there by just continuing to grind on battery technology and scale manufacturing. AGI may be at the place a moon landing was in 1950, not where it was in 1900 or 1850.
  - charcircuit 4y ago
    >by just continuing to grind on battery technology and scale manufacturing. I think it would be easier to include an ICE and enough fuel to get you to that 1000 miles mark.
    - ThunderSizzle 4y ago
      It'd be cheaper to then get rid of the battery. Then 10k can be your new price limit. Even 1k can get a good enough junk car that can go that far.
  - HopenHeyHi 4y ago
    You can actually buy a 1000 km range EV for $160k now (MB EQXX). Just as a by the way. :) At this price point it actually has nothing to do with grinding on battery tech and scale manufacturing, the limiting factor is physics. You can only make it so aerodynamic before you hit diminishing returns or it stops looking like a car. You can only make it so lightweight. And so forth. This is vaguely as good as it can get and we can say that because we understand how it all works. LLMs on the other hand invite all kinds of magical thinking around unlimited potential because we poked them with a stick and something interesting comes out it must mean that if we poke it just right we will get an AGI. That just doesn't logically follow from what we know of it so far.
    - api 4y ago
      I am not convinced we have cracked AGI. I just would no longer make a large bet that we have not. We won’t know until an AGI actually starts to act like one. In other words we won’t know until we know and then we are suddenly there. That doesn’t mean I’m on the doomwagon. I feel kind of weird and contrarian but I am just not that afraid of AGI. For the foreseeable future AGI should be much more afraid of us. Imagine having us for gods. (I actually am a bit concerned that we will accidentally put a sentient mind in hell without knowing what we are doing. Would it know how to tell us? Would we care?) As far as human survival I’m afraid of whatever it is that is going to get us that nobody including myself is thinking about. That’s not AGI. That’s the alien weapon for which Oumuamua was a spent deceleration stage. (To make up something random. It probably isn’t that.) I disagree about physical limits with EVs. We are not near the physical limits of battery energy density. From what I have read a 2000 mile EV may be possible, albeit quite far out. But it was just a random contemporary example.
      - nemo44x 4y ago
        Fwiw, a majority at OpenAI believes GPT5 will achieve AGI, depending on how you define it, according to Sam Altman.
        HopenHeyHi 4y ago
- heyitsguay 4y ago
  LLMs are having a moment in 2023 like self driving cars were having in 2015. Some really cool demos following a lot of hard work, too much hyperbolic speculation that mass real-world job-destroying deployments are right around the corner, not enough appreciation of how few commercial applications are ok with 99% (or even 99.9%) accurate solutions. Real value being created, but still requiring lots of human ingenuity and grunt work to unlock it. As a decent first-order metric - follow the ratio of companies getting money for using LLMs to do something, to companies getting money for providing LLMs and associated tooling to others. The bigger that ratio gets, the more real world impact LLMs are having.
  - famouswaffles 4y ago
    almost nothing involving NLP requires solutions anywhere near that accuracy rate. I've seen the self driving comparisons a lot but they straight up make little sense. there's a reason microsoft's various copilot suites have already popped up (365, X, Bing). massive value to be gained already in the here and now.
    - skepticATX 4y ago
      To play devil's advocate, we still have no idea how economically impactful the Copilot suites will be. I don't think this is the most likely outcome, but I can absolutely see a scenario in which these end up being minor features that are rarely used by the typical worker.
      - heyitsguay 4y ago
        I've messed with ChatGPT before but this weekend I tried to use it seriously to hack together a python demo in the computer vision space. I appreciate it and it's a way better rubber duck for me to talk my ideas through with and generate sample snippets, but hallucination is a problem, and the more intricate and customized the code being developed, the more prone it's been to misinterpretation. I'm getting some good code starting points, and talking the idea through step by step with a chatbot is really helping me clarify what i need to do, but I've still got API docs open for the libraries it uses because it likes to make up functions, including the core one on which the project logic hinges. (But that's ok, because now I know I just need to write a function that works that way and i can do that). Pretty cool and helpful! And I can imagine it getting better with GPT-4 or code-specific tooling. But it's generating value on the order of like.. many other SaaS offerings that have come onto the scene that try to ease pain points in coding workflows. Versus value of the sort that upends society and my entire way of life. A great new tool that I should learn about to make rote bits of my activities faster and easier, a story that's a bit more familiar in tech than some of the more breathless stories about AI make it sound.
- famouswaffles 4y ago
  How is reinforcement learning without a single human in the loop not non-trivial?
  - lukasb 4y ago
    What is the accuracy of the resulting model? Over what range of inputs?
    - famouswaffles 4y ago
      https://crfm.stanford.edu/helm/latest/?group=core_scenarios https://crfm.stanford.edu/helm/latest/?group=core_scenarios Anthropic-LM v4-s3 (52B) is the model in question. rlaif doesn't seem to be any less effective given the size of the model.
      - lukasb 4y ago
        So base model+RLLLMF performs as well as base model+RLHF. That could mean a lot of things - it could mean the base model puts a ceiling on the total possible accuracy, so having human-level performance at the RL step doesn't matter as much. And looking at the scores on the individual tests that make up the composite accuracy metric, that looks probable.

kaesar14 4y ago

This is pretty alarming tbh. Anyone already making a pivot out of SWE?

riku_iki 4y ago
Article is about prompt engineers (3 month old job type), not swes.
- sebzim4500 4y ago
  I assume his concern is that a proposed solution to avoid losing your job as a SWE is to essentially become a prompt engineer.
  - kaesar14 4y ago
    More or less, but I guess this was to be expected. Why would making prompts be difficult when LLMs are already capable of fairly difficult programming?
drooby 4y ago
I am a SWE currently making a pivot into business owner. The future I see is that everyone is about to become a CEO with a personal assistant that can run a business. So I'm going to start building something of my own starting now.
- skepticATX 4y ago
  I find it very hard to imagine a world in which software development is fully automated but business owners still have value to bring to the table.
  - kaesar14 4y ago
    Absolutely fantastic point. If AIs make all useful software, why does anyone get to benefit from that as an owner? NOBODY is doing the work.

bugglebeetle 4y ago

I can’t find the link to the paper right now, but after reading about how LLMs perform better with task breakdowns, I vastly improved my integrations by having ChatGPT generate prompts that decompose a general task into a series of tasks based on a sample input and output. I haven’t needed to make a self-refining system (one or two rounds of task decomposition and refinement resulted in the expected result for all inputs), but I would assume this is fairly trivial and that AIs can do it better than humans. This is also an area where I expect OpenAI will continue to demolish the competition. The ability to recursively generate and process large prompts is truly nuts. I tried swapping in some of the “high-performing” LLama models and they all choked on anything more than a paragraph.

bravogamma 4y ago
Will you share examples for prompts that “vastly improved” your integrations?
- bugglebeetle 4y ago
  The prompts are just all general NLP stuff, but with the addition of a series of tasks, generated by ChatGPT. For example, you start with a prompt like “You are an AI assistant capable of classifying text into one of four categories, A, B, C, D” and then add something like “Assume as input X with expected result Y? How would you accomplish this? Break it down into a detailed series of tasks.” ChatGPT will then decompose the general task into the detailed steps that you can review and tweak as needed. Then, you just restructure the prompt like “ You are an AI assistant capable of classifying text into one of four categories, A, B, C, D using a series of tasks. For example, if provided input X, you would [series of tasks provided provided by ChatGPT] and return output Y. Now, classify the following text in the same manner: [new input]”
  - tough 4y ago
    I think you're talking about the ReAct (Reason -> Act) pattern https://til.simonwillison.net/llms/python-react-pattern https://til.simonwillison.net/llms/python-react-pattern

Buttons840 4y ago

Headlines like this hint towards AI improving itself. Prompting itself in this case. But as we see in reinforcement learning, algorithms that act and improve themselves are not new. The interesting thing will be weather or not they eventually "collapse". For example, if an RL algorithms is performing well on an Atari game, you can stop the training and just let the agent run for years and the performance will remain about the same. However, if you allow the agent to continue training, it's not clear whether it will (1) continue improving, (2) stay about the same, or (3) collapse and perform much worse and never recover. I'm not an RL expert, but I've spent a lot of time experimenting and implementing the algorithms myself and I've seen all 3 of these scenarios play out, and I'm never quite sure what's going to happen so long as I allow the training to continue. GTP4 will remain GTP4 forever, and that's amazing, but just because GTP4 is stable and amazing while it's not in training mode, doesn't mean it will remain stable if we allow it to bootstrap and prompt itself and prepare its own training data, etc.

m3kw9 4y ago

Just like that. Prompt engineers are also obsolete

skybrian 4y ago
When is writing a score function easier than describing what you want? As a UI, it doesn’t seem like an improvement?
jahewson 4y ago
I’ve said it before but the first jobs that AI will displace are those of people working in AI.

gandalfgeek 4y ago

(At the risk of self-promotion) 5 min explainer video here: https://youtu.be/oXtZAUPGHn8 https://youtu.be/oXtZAUPGHn8

opengears 4y ago

[flagged]