4 ms·

It's magical thinking all the way down. Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "stick

It's magical thinking all the way down. Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space.

nonethewiser 1y ago
>Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space. I dont quite follow. Prompts and contexts are different things. Sure, you can get thing into contexts with prompts but that doesn't mean they are entirely the same. You could have a long running conversation with a lot in the context. A given prompt may work poorly, whereas it would have worked quite well earlier. I don't think this difference is purely semantic. For whatever it's worth I've never liked the term "prompt engineering." It is perhaps the quintessential example of overusing the word engineering.
- ffsm8 1y ago
  Yeah, if anything it should be called an art. The term engineering makes little sense in this context, but really... Did it make sense for eg "QA Engineer" and all the other jobs we tacked it on, too? I don't think so, so it's kinda arguing after we've been misusing the term for well over 10 yrs
  - groestl 1y ago
    Well, to get the right thing into the context in a performant way when you dealing with a huge dataset is definitely engineering.
    - shakna 1y ago
      Engineering tends to mean "the application of scientific and mathematical principles to practical ends". I'm not sure there's much scientific or mathematical about guessing how a non-deterministic system will behave.
      - groestl 1y ago
        Funny how you use a scientific term to discredit applied statistics. I've built useful non-deterministic systems many times and they had nothing to do with AI. Also, particle physics would like to have a word with you.
        shakna 1y ago
        Guessing, how a non-deterministic system would behave. Statistics isn't guessing. But it is guessing when the confidence interval is unknowable and constantly shifting. We're not talking relativity, we're talking about throwing pancakes at a wall to tell if there's a person behind it.
        deleted 1y ago
        [deleted]
      - SonOfLilit 1y ago
        The moment you start building evaluation pipelines and running experiments to validate your ideas it stops being guessing
        simonw 1y ago
        Right: for me that's when "prompt engineering"/"context engineering" start to earn the "engineering" suffix: when people start being methodical and applying techniques like evals.
        passwordqwe 1y ago
        Relevant XKCD: https://xkcd.com/397/ https://xkcd.com/397/ About if it's science or not, the difference is testing it through experiment.
        deleted 1y ago
        [deleted]
- Turskarama 1y ago
  Both the context and the prompt are just part of the same input. To the model there is no difference, the only difference is the way the user feeds that input to the model. You could in theory feed the context into the model as one huge prompt.
  - __loam 1y ago
    Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.
    - buffzebra 1y ago
      “It’s all just tokens in the context window” = “it’s all just fundamental particles,” I think. True, but reductive. Seems key that dude is talking about agentic AI not just chat. I’d revisit the email example in the post.
    - StevenWaterman 1y ago
      Yep, every AI call is essentially just asking it to predict what the next word is after: <system> You are a helpful assistant. </system> <user> Why is the sky blue? </user> <assistant> Because of Rayleigh scattering. The blue light refracts more. </assistant> <user> Why is it red at sunset then? </user> <assistant> And we keep repeating that until the next word is `</assistant>`, then extract the bit in between the last assistant tags, and return it. The AI has been trained to look at `<user>` differently to `<system>`, but they're not physically different. It's all prompt, it can all be engineered. Hell, you can even get a long way by pre-filling the start of the Assistant response. Usually works better than a system message. That's prompt engineering too.
      - Terr_ 1y ago
        Yeah, ultimately it's Make Document Longer machine, and in many cases it's a hidden mad-libs script behind the scenes, where your question becomes "Next the User said", and some regular code is looking for "Next the Computer said" and "performing" it at you. In other words, there's a deliberate illusion going on where we are encouraged to believe that generating a document about a character is the same as that character being a real entity.
- pennaMan 1y ago
  I always used "prompting" to mean "providing context" in genral not necesarlly just clever instructions like people seem to be using the term. And yes, I view clever instructions like "great grandma's last wish" still as just providing context. >A given prompt may work poorly, whereas it would have worked quite well earlier. The context is not the same! Of course the "prompt" (clever last sentence you just added to the context) is not going to work "the same". The model has a different context now.
- sethammons 1y ago
  "Context Crafting"
surecoocoocoo 1y ago
We used to define a specification. In other words; context. But that was like old man programming. As the laws of physics changed between 1970 and 2009.
- prmph 1y ago
  Is this Haiku?
ironmagma 1y ago
What is all software but tinkering? I mean this not as an insult to software dev but to work generally. It’s all play in the end.
- 8n4vidtmkvmk 1y ago
  I don't buy this. With software engineering you can generally make incremental progress towards your goal. Yes, sometimes you have to scrap stuff, but usually not the entire thing because an LLM spout out pure nonsense.
belter 1y ago
Got it...updating CV to call myself a VibeOps Engineer in a team of Context Engineers...A few of us were let go last quarter, as they could only do Prompt Engineering.
tootie 1y ago
You say "magic" I say "heuristic"

It's magical thinking all the way down. Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space.

nonethewiser 1y ago

>Whether they call it now "prompt" or "context" engineering because it's the same tinkering to find something that "sticks" in non-deterministic space. I dont quite follow. Prompts and contexts are different things. Sure, you can get thing into contexts with prompts but that doesn't mean they are entirely the same. You could have a long running conversation with a lot in the context. A given prompt may work poorly, whereas it would have worked quite well earlier. I don't think this difference is purely semantic. For whatever it's worth I've never liked the term "prompt engineering." It is perhaps the quintessential example of overusing the word engineering.

ffsm8 1y ago
Yeah, if anything it should be called an art. The term engineering makes little sense in this context, but really... Did it make sense for eg "QA Engineer" and all the other jobs we tacked it on, too? I don't think so, so it's kinda arguing after we've been misusing the term for well over 10 yrs
- groestl 1y ago
  Well, to get the right thing into the context in a performant way when you dealing with a huge dataset is definitely engineering.
  - shakna 1y ago
    Engineering tends to mean "the application of scientific and mathematical principles to practical ends". I'm not sure there's much scientific or mathematical about guessing how a non-deterministic system will behave.
    - groestl 1y ago
      Funny how you use a scientific term to discredit applied statistics. I've built useful non-deterministic systems many times and they had nothing to do with AI. Also, particle physics would like to have a word with you.
      - shakna 1y ago
        Guessing, how a non-deterministic system would behave. Statistics isn't guessing. But it is guessing when the confidence interval is unknowable and constantly shifting. We're not talking relativity, we're talking about throwing pancakes at a wall to tell if there's a person behind it.
        deleted 1y ago
        [deleted]
    - SonOfLilit 1y ago
      The moment you start building evaluation pipelines and running experiments to validate your ideas it stops being guessing
      - simonw 1y ago
        Right: for me that's when "prompt engineering"/"context engineering" start to earn the "engineering" suffix: when people start being methodical and applying techniques like evals.
        passwordqwe 1y ago
        Relevant XKCD: https://xkcd.com/397/ https://xkcd.com/397/ About if it's science or not, the difference is testing it through experiment.
        deleted 1y ago
        [deleted]
Turskarama 1y ago
Both the context and the prompt are just part of the same input. To the model there is no difference, the only difference is the way the user feeds that input to the model. You could in theory feed the context into the model as one huge prompt.
- __loam 1y ago
  Sometimes I wonder if LLM proponents even understand their own bullshit. It's all just tokens in the context window right? Aren't system prompts just tokens that stay appended to the front of a conversation? They're going to keep dressing this up six different ways to Sunday but it's always just going to be stochastic token prediction.
  - buffzebra 1y ago
    “It’s all just tokens in the context window” = “it’s all just fundamental particles,” I think. True, but reductive. Seems key that dude is talking about agentic AI not just chat. I’d revisit the email example in the post.
  - StevenWaterman 1y ago
    Yep, every AI call is essentially just asking it to predict what the next word is after: <system> You are a helpful assistant. </system> <user> Why is the sky blue? </user> <assistant> Because of Rayleigh scattering. The blue light refracts more. </assistant> <user> Why is it red at sunset then? </user> <assistant> And we keep repeating that until the next word is `</assistant>`, then extract the bit in between the last assistant tags, and return it. The AI has been trained to look at `<user>` differently to `<system>`, but they're not physically different. It's all prompt, it can all be engineered. Hell, you can even get a long way by pre-filling the start of the Assistant response. Usually works better than a system message. That's prompt engineering too.
    - Terr_ 1y ago
      Yeah, ultimately it's Make Document Longer machine, and in many cases it's a hidden mad-libs script behind the scenes, where your question becomes "Next the User said", and some regular code is looking for "Next the Computer said" and "performing" it at you. In other words, there's a deliberate illusion going on where we are encouraged to believe that generating a document about a character is the same as that character being a real entity.
pennaMan 1y ago
I always used "prompting" to mean "providing context" in genral not necesarlly just clever instructions like people seem to be using the term. And yes, I view clever instructions like "great grandma's last wish" still as just providing context. >A given prompt may work poorly, whereas it would have worked quite well earlier. The context is not the same! Of course the "prompt" (clever last sentence you just added to the context) is not going to work "the same". The model has a different context now.
sethammons 1y ago
"Context Crafting"

surecoocoocoo 1y ago

We used to define a specification. In other words; context. But that was like old man programming. As the laws of physics changed between 1970 and 2009.

prmph 1y ago
Is this Haiku?

ironmagma 1y ago

What is all software but tinkering? I mean this not as an insult to software dev but to work generally. It’s all play in the end.

8n4vidtmkvmk 1y ago
I don't buy this. With software engineering you can generally make incremental progress towards your goal. Yes, sometimes you have to scrap stuff, but usually not the entire thing because an LLM spout out pure nonsense.

belter 1y ago

Got it...updating CV to call myself a VibeOps Engineer in a team of Context Engineers...A few of us were let go last quarter, as they could only do Prompt Engineering.

tootie 1y ago

You say "magic" I say "heuristic"