4 ms·

“A month-long skill” after which it won’t be a thing anymore, like so many other.

simonw 1y ago
Most of the LLM prompting skills I figured out ~three years ago are still useful to me today. Even the ones that I've dropped are useful because I know that things that used to be helpful aren't helpful any more, which helps me build an intuition for how the models have improved over time.
- refulgentis 1y ago
  I agree with you, but would echo OP's concern, in a way that makes me feel like a party pooper, but, is open about what I see us all expressing squeamish-ness about. It is somewhat bothersome to have another buzz phrase. I don't why we are doing this, other than there was a Xeet from the Shopify CEO, QT'd approvingly by Karpathy, then its written up at length, and tied to another set of blog posts. To wit, it went from "buzzphrase" to "skill that'll probably be useful in 3 years still" over the course of this thread. Has it even been a week since the original tweet? There doesn't seem to be a strong foundation here, but due to the reach potential of the names involved, and their insistence on this being a thing while also indicating they're sheepish it is a thing, it will now be a thing. Smacks of a self-aware version of Jared Friedman's tweet re: watching the invention of "Founder Mode" was like a startup version of the Potsdam Conference. (which sorted out Earth post-WWII. and he was not kidding. I could not even remember the phrase for the life of me. Lasted maybe 3 months?)
  - simonw 1y ago
    The way I see it we're trying to rebrand because the term "prompt engineering" got redefined to mean "typing prompts full of stupid hacks about things like tipping and dead grandmas into a chatbot".
    - Dylan16807 1y ago
      If that's what it takes to make good results, then it's respectable work even if the details are stupid.
    - joe5150 1y ago
      It helps that the rebrand may lead some people to believe that there are actually new and better inputs into the system rather than just more elaborate sandcastles built in someone else's sandbox.
  - dbreunig 1y ago
    Sometimes buzzwords turn out to be mirages that disappear in a few weeks, but often they stick around. I find they takeoff when someone crystallizes something many people are thinking about internally, and don’t realize everyone else is having similar thoughts. In this example, I think the way agent and app builders are wrestling with LLMs is fundamentally different than chatbots users (it’s closer to programming), and this phrase resonates with that crowd. Here’s an earlier write up on buzzwords: https://www.dbreunig.com/2020/02/28/how-to-build-a-buzzword.html https://www.dbreunig.com/2020/02/28/how-to-build-a-buzzword....
    - refulgentis 1y ago
      I agree - what distinguishes this is how rushed and self-aware it is. It is being pushed top down, sheepishly. EDIT: Ah, you also wrote the blog posts tied to this. It gives 0 comfort that you have a blog post re: building buzz phrases in 2020, rather, it enhances the awkward inorganic rush people are self-aware of.
      - dbreunig 1y ago
        I studied linguistic anthropology, in addition to CS. Been at it since 2002. And I wrote the first post before the meme.
        refulgentis 1y ago
        I've read these ideas a 1000 times, I thought it was the most beautiful core of the "Sparks of AGI" paper. (6.2) We should be able to name the source of this sheepishness and have fun with that we are all things at once: you can be a viral hit 2002 super PhD with expertise in all areas involved in this topic that has brought pop attention onto something important, and yet, the hip topic you feel centered on can also make people's eyes roll temporarily. You're doing God's work. The AI = F(C) thing is really important. Its just, in the short term, it will feel like a buzzword. This is much more about me playing with, what we can reduce to, the "get off my lawn!" take. I felt it interesting to voice because it is a consistent undercurrent in the discussion and also leads to observable absurdities when trying to describe it. It is not questioning you, your ideas, or work. It has just come about at a time when things become hyperreal hyperquickly and I am feeling old.
- dbreunig 1y ago
  While researching the above posts Simon linked, I was struck by how many of these techniques came from the pre-ChatGPT era. NLP researchers have been dealing with this for awhile.
orbital-decay 1y ago
Many people figured it out two-three years ago when AI-assisted coding basically wasn't a thing, and it's still relevant and will stay relevant. These are fundamental principles, all big models work similarly, not just transformers and not just LLMs. However, many fundamental phenomena are missing from the "context engineering" scope, so neither context engineering nor prompt engineering are useful terms.
coldtea 1y ago
What exactly month-long AI skills of 2023 AI are obsolete now? Surely not prompt engineering itself, for example.
- bird0861 1y ago
  Persona prompting. (Unless the persona is the point as in role-playing.)
tptacek 1y ago
If you're not writing your own agents, you can skip this skill.
- anilgulecha 1y ago
  Are you sure? Looking forward - AI is going to be so pervasively used, that understanding what information is to be input will be a general skill. What we've been calling "prompt engineering" - the better ones were actually doing context engineering.
  - tptacek 1y ago
    If you're doing context engineering, you're writing an agent. It's mostly not the kind of stuff you can do from a web chat textarea.

Most of the LLM prompting skills I figured out ~three years ago are still useful to me today. Even the ones that I've dropped are useful because I know that things that used to be helpful aren't helpful any more, which helps me build an intuition for how the models have improved over time.

refulgentis 1y ago
I agree with you, but would echo OP's concern, in a way that makes me feel like a party pooper, but, is open about what I see us all expressing squeamish-ness about. It is somewhat bothersome to have another buzz phrase. I don't why we are doing this, other than there was a Xeet from the Shopify CEO, QT'd approvingly by Karpathy, then its written up at length, and tied to another set of blog posts. To wit, it went from "buzzphrase" to "skill that'll probably be useful in 3 years still" over the course of this thread. Has it even been a week since the original tweet? There doesn't seem to be a strong foundation here, but due to the reach potential of the names involved, and their insistence on this being a thing while also indicating they're sheepish it is a thing, it will now be a thing. Smacks of a self-aware version of Jared Friedman's tweet re: watching the invention of "Founder Mode" was like a startup version of the Potsdam Conference. (which sorted out Earth post-WWII. and he was not kidding. I could not even remember the phrase for the life of me. Lasted maybe 3 months?)
- simonw 1y ago
  The way I see it we're trying to rebrand because the term "prompt engineering" got redefined to mean "typing prompts full of stupid hacks about things like tipping and dead grandmas into a chatbot".
  - Dylan16807 1y ago
    If that's what it takes to make good results, then it's respectable work even if the details are stupid.
  - joe5150 1y ago
    It helps that the rebrand may lead some people to believe that there are actually new and better inputs into the system rather than just more elaborate sandcastles built in someone else's sandbox.
- dbreunig 1y ago
  Sometimes buzzwords turn out to be mirages that disappear in a few weeks, but often they stick around. I find they takeoff when someone crystallizes something many people are thinking about internally, and don’t realize everyone else is having similar thoughts. In this example, I think the way agent and app builders are wrestling with LLMs is fundamentally different than chatbots users (it’s closer to programming), and this phrase resonates with that crowd. Here’s an earlier write up on buzzwords: https://www.dbreunig.com/2020/02/28/how-to-build-a-buzzword.html https://www.dbreunig.com/2020/02/28/how-to-build-a-buzzword....
  - refulgentis 1y ago
    I agree - what distinguishes this is how rushed and self-aware it is. It is being pushed top down, sheepishly. EDIT: Ah, you also wrote the blog posts tied to this. It gives 0 comfort that you have a blog post re: building buzz phrases in 2020, rather, it enhances the awkward inorganic rush people are self-aware of.
    - dbreunig 1y ago
      I studied linguistic anthropology, in addition to CS. Been at it since 2002. And I wrote the first post before the meme.
      - refulgentis 1y ago
        I've read these ideas a 1000 times, I thought it was the most beautiful core of the "Sparks of AGI" paper. (6.2) We should be able to name the source of this sheepishness and have fun with that we are all things at once: you can be a viral hit 2002 super PhD with expertise in all areas involved in this topic that has brought pop attention onto something important, and yet, the hip topic you feel centered on can also make people's eyes roll temporarily. You're doing God's work. The AI = F(C) thing is really important. Its just, in the short term, it will feel like a buzzword. This is much more about me playing with, what we can reduce to, the "get off my lawn!" take. I felt it interesting to voice because it is a consistent undercurrent in the discussion and also leads to observable absurdities when trying to describe it. It is not questioning you, your ideas, or work. It has just come about at a time when things become hyperreal hyperquickly and I am feeling old.
dbreunig 1y ago
While researching the above posts Simon linked, I was struck by how many of these techniques came from the pre-ChatGPT era. NLP researchers have been dealing with this for awhile.

orbital-decay 1y ago

Many people figured it out two-three years ago when AI-assisted coding basically wasn't a thing, and it's still relevant and will stay relevant. These are fundamental principles, all big models work similarly, not just transformers and not just LLMs. However, many fundamental phenomena are missing from the "context engineering" scope, so neither context engineering nor prompt engineering are useful terms.

coldtea 1y ago

What exactly month-long AI skills of 2023 AI are obsolete now? Surely not prompt engineering itself, for example.

bird0861 1y ago
Persona prompting. (Unless the persona is the point as in role-playing.)

tptacek 1y ago

If you're not writing your own agents, you can skip this skill.

anilgulecha 1y ago
Are you sure? Looking forward - AI is going to be so pervasively used, that understanding what information is to be input will be a general skill. What we've been calling "prompt engineering" - the better ones were actually doing context engineering.
- tptacek 1y ago
  If you're doing context engineering, you're writing an agent. It's mostly not the kind of stuff you can do from a web chat textarea.