4 ms·

This is the only approach that seems even remotely reasonable “Prompt engineering” just seems dumb as hell. It’s literally just an imprecise nondeterministic p

by lkjdsklf 10mo ago

This is the only approach that seems even remotely reasonable “Prompt engineering” just seems dumb as hell. It’s literally just an imprecise nondeterministic programming language. Before a couple years so, we all would have said that was a bad language and moved on.

BoorishBears 10mo ago
A few years ago we didn't have an imprecise nondeterministic programming language that would allow your mom to achieve SOTA results on a wide range of NLP tasks by asking nicely, or I'm sure people would have taken it. I think a lot of prompt engineering is voodoo, but it's not all baseless: a more formal way to look at it is aligning your task with the pre-training and post-training of the model. The whole "it's a bad language" refrain feels half-baked when most of us use relatively high level languages on non-realtime OSes that obfuscate so much that they might as well be well worded prompts compared to how deterministic the underlying primitives they were built on are... at least until you zoom in too far.
- jakelazaroff 10mo ago
  Do you mean, like, scripting languages? Are the underlying primitives C and machine language? "Might as well be well worded prompts" is the overstatement of the century; any given scripting language is far closer to those underlying layers than it is to using natural language with LLMs.
  - BoorishBears 10mo ago
    Sure doesn't seem like it. https://x.com/jarredsumner/status/1999317065237512224 https://x.com/jarredsumner/status/1999317065237512224 And forget scripting languages, take a C program that writes a string to disk and reads it back. How many times longer does it get the moment we have to ensure the string was actually committed to non-volatile NAND and actually read back? 5x? 10x? Is it even doable if we have to support arbitrary consumer hardware?
    - lkjdsklf 10mo ago
      You're stretching really hard here to try and rationalize your position First of all, I pick the hardware I support and the operating systems. I can make those things requirements when they are required. But when you boil down your argument, it's that because one thing may introduce non-determinism, then any degree of non-determinism is acceptable. At that point we don't even need LLMs. We can just have the computer do random things. It's just a rehash of the infinite monkeys with infinite type writers which is ridiculous
      - BoorishBears 10mo ago
        No the point was quite clear: > A few years ago we didn't have an imprecise nondeterministic programming language that would allow your mom to achieve SOTA results on a wide range of NLP tasks by asking nicely, or I'm sure people would have taken it. But that (accurate) point makes your point invalid, so you'd rather focus on the dressing.
        jakelazaroff 10mo ago
        We still don't have that programming language (although "SOTA" and "wide range of NLP tasks" are vague enough that you can probably move the goalposts into field goal range).
        BoorishBears 10mo ago
        This comment is written way too adversarially for someone who doesn't know what NLP is.
    - deleted 10mo ago
      [deleted]
- ehnto 10mo ago
  I don't buy your past paragraph at all I am afraid. Coding langues, even high level ones, are built upon foundations of determism and they are concise and precise. A short way to describe very precisely, a bunch of rules and state. Prompting is none of those things. It is a ball of math we can throw words into, and it approximates meaning and returns an output with randomness built in. That is incredible, truly, but it is not a programming language.
  - BoorishBears 10mo ago
    Eh, how modern technology works is not really the part I'm selling: that's just how it works. Coding languages haven't been describing even a fraction of the rules and state they encapsulate since what? Punch cards? It wasn't long until we started to rely on exponential number of layered abstractions to do anything useful with computers, and very quickly we traded precision and determinism for benefits like being concise and easier to reason about. - But also, the context here was someone calling prompting a "imprecise nondeterministic programming language": obviously their bone is the "imprecise nondeterministic" part, not distilling what defines a programming language. I get it doesn't feel warm and fuzzy to the average engineer, but realistically we were hand engineering solutions with "precise deterministic programming languages", they were similarly probabilistic, and they performed worse.
    - lifeformed 10mo ago
      Name a single programming language that is probabilistic in any way?
      - BoorishBears 10mo ago
        - A text prompt isn't probabilistic, the output is. - https://labs.oracle.com/pls/apex/f?p=LABS:0:5033606075766:APPLICATION_PROCESS=GETDOC_INLINE:::DOC_ID:4510 https://labs.oracle.com/pls/apex/f?p=LABS:0:5033606075766:AP... - https://en.wikipedia.org/wiki/Stan_(software) https://en.wikipedia.org/wiki/Stan_(software) - https://en.wikipedia.org/wiki/Probabilistic_programming https://en.wikipedia.org/wiki/Probabilistic_programming I explained in the most clear language possible why a fixation on the "programming language" part of the original comment is borderline non-sequitur. But if you're insistent on railroading the conversation regardless... at least try to be good at it, no?
- lkjdsklf 10mo ago
  > nondeterministic programming language that would allow your mom to achieve SOTA results I actually think it's great for giving non-programmers the ability to program to solve basic problems. That's really cool and it's pretty darn good at it. I would refute that you get SOTA results. That has never been my personal experience. Given that we don't see a large increase in innovative companies spinning up now that this technology is a few years old, I doubt it's the experience of most users. > The whole "it's a bad language" refrain feels half-baked when most of us use relatively high level languages on non-realtime OSes that obfuscate so much that they might as well be well worded prompts compared to how deterministic the underlying primitives they were built on are... at least until you zoom in too far. Obfuscation and abstraction are not the same thing. The other core difference is the precision and the determinism both of which are lacking with LLMs.
potatoman22 10mo ago
It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled (temp > 0) and input is varied. Using the right prompt makes the model perform better on average, so it's not completely dumb. I like task vectors and soft prompts because I think they show how prompt engineering is cool and useful. https://arxiv.org/pdf/2310.15916 https://arxiv.org/pdf/2310.15916 https://huggingface.co/docs/peft/conceptual_guides/prompting https://huggingface.co/docs/peft/conceptual_guides/prompting
- BoiledCabbage 10mo ago
  > It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled Are you not aware the random sampling makes something non-deterministic?
  - rTX5CMRXIfFG 10mo ago
    I don’t believe that when he wrote that, he was using his own intelligence
    - potatoman22 10mo ago
      At least I'm adding to the discussion.
      - rTX5CMRXIfFG 10mo ago
        People who disagree with you add to the discussion too
  - potatoman22 10mo ago
    I'm saying LLMs are deterministic and because of that, prompt engineering can be effective. You knew what I was trying to say, but chose to ignore it. You should follow the HN Guidelines. I'm trying to have a discussion, not a snarkfest. > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
theshrike79 10mo ago
You know what else is nondeterministic? "Project Management" You tell humans to make code, they make code. How do you check it's correct and does what it should?
- abenga 10mo ago
  Humans have agency and accountability, duty of care, etc.
  - theshrike79 10mo ago
    How does a random programmer have "duty of care"? They're paid to ship a feature, why would they spend extra time making something perfect? If it breaks they just fix it - and get paid doing it. Not everyone writing code in a corporation is a Code Artesan who handcrafts every single character with thought and precision like an old Japanese craftsman. They just write code, it does about what it should and push a PR branch. Then you grab the next task from the corporate Jira project.
    - abenga 10mo ago
      > How does a random programmer have "duty of care"? Are there jobs that don't care if you don't take due care when writing code? I'd like me one of those.

BoorishBears 10mo ago

A few years ago we didn't have an imprecise nondeterministic programming language that would allow your mom to achieve SOTA results on a wide range of NLP tasks by asking nicely, or I'm sure people would have taken it. I think a lot of prompt engineering is voodoo, but it's not all baseless: a more formal way to look at it is aligning your task with the pre-training and post-training of the model. The whole "it's a bad language" refrain feels half-baked when most of us use relatively high level languages on non-realtime OSes that obfuscate so much that they might as well be well worded prompts compared to how deterministic the underlying primitives they were built on are... at least until you zoom in too far.

jakelazaroff 10mo ago
Do you mean, like, scripting languages? Are the underlying primitives C and machine language? "Might as well be well worded prompts" is the overstatement of the century; any given scripting language is far closer to those underlying layers than it is to using natural language with LLMs.
- BoorishBears 10mo ago
  Sure doesn't seem like it. https://x.com/jarredsumner/status/1999317065237512224 https://x.com/jarredsumner/status/1999317065237512224 And forget scripting languages, take a C program that writes a string to disk and reads it back. How many times longer does it get the moment we have to ensure the string was actually committed to non-volatile NAND and actually read back? 5x? 10x? Is it even doable if we have to support arbitrary consumer hardware?
  - lkjdsklf 10mo ago
    You're stretching really hard here to try and rationalize your position First of all, I pick the hardware I support and the operating systems. I can make those things requirements when they are required. But when you boil down your argument, it's that because one thing may introduce non-determinism, then any degree of non-determinism is acceptable. At that point we don't even need LLMs. We can just have the computer do random things. It's just a rehash of the infinite monkeys with infinite type writers which is ridiculous
    - BoorishBears 10mo ago
      No the point was quite clear: > A few years ago we didn't have an imprecise nondeterministic programming language that would allow your mom to achieve SOTA results on a wide range of NLP tasks by asking nicely, or I'm sure people would have taken it. But that (accurate) point makes your point invalid, so you'd rather focus on the dressing.
      - jakelazaroff 10mo ago
        We still don't have that programming language (although "SOTA" and "wide range of NLP tasks" are vague enough that you can probably move the goalposts into field goal range).
        BoorishBears 10mo ago
        This comment is written way too adversarially for someone who doesn't know what NLP is.
  - deleted 10mo ago
    [deleted]
ehnto 10mo ago
I don't buy your past paragraph at all I am afraid. Coding langues, even high level ones, are built upon foundations of determism and they are concise and precise. A short way to describe very precisely, a bunch of rules and state. Prompting is none of those things. It is a ball of math we can throw words into, and it approximates meaning and returns an output with randomness built in. That is incredible, truly, but it is not a programming language.
- BoorishBears 10mo ago
  Eh, how modern technology works is not really the part I'm selling: that's just how it works. Coding languages haven't been describing even a fraction of the rules and state they encapsulate since what? Punch cards? It wasn't long until we started to rely on exponential number of layered abstractions to do anything useful with computers, and very quickly we traded precision and determinism for benefits like being concise and easier to reason about. - But also, the context here was someone calling prompting a "imprecise nondeterministic programming language": obviously their bone is the "imprecise nondeterministic" part, not distilling what defines a programming language. I get it doesn't feel warm and fuzzy to the average engineer, but realistically we were hand engineering solutions with "precise deterministic programming languages", they were similarly probabilistic, and they performed worse.
  - lifeformed 10mo ago
    Name a single programming language that is probabilistic in any way?
    - BoorishBears 10mo ago
      - A text prompt isn't probabilistic, the output is. - https://labs.oracle.com/pls/apex/f?p=LABS:0:5033606075766:APPLICATION_PROCESS=GETDOC_INLINE:::DOC_ID:4510 https://labs.oracle.com/pls/apex/f?p=LABS:0:5033606075766:AP... - https://en.wikipedia.org/wiki/Stan_(software) https://en.wikipedia.org/wiki/Stan_(software) - https://en.wikipedia.org/wiki/Probabilistic_programming https://en.wikipedia.org/wiki/Probabilistic_programming I explained in the most clear language possible why a fixation on the "programming language" part of the original comment is borderline non-sequitur. But if you're insistent on railroading the conversation regardless... at least try to be good at it, no?
lkjdsklf 10mo ago
> nondeterministic programming language that would allow your mom to achieve SOTA results I actually think it's great for giving non-programmers the ability to program to solve basic problems. That's really cool and it's pretty darn good at it. I would refute that you get SOTA results. That has never been my personal experience. Given that we don't see a large increase in innovative companies spinning up now that this technology is a few years old, I doubt it's the experience of most users. > The whole "it's a bad language" refrain feels half-baked when most of us use relatively high level languages on non-realtime OSes that obfuscate so much that they might as well be well worded prompts compared to how deterministic the underlying primitives they were built on are... at least until you zoom in too far. Obfuscation and abstraction are not the same thing. The other core difference is the precision and the determinism both of which are lacking with LLMs.

potatoman22 10mo ago

It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled (temp > 0) and input is varied. Using the right prompt makes the model perform better on average, so it's not completely dumb. I like task vectors and soft prompts because I think they show how prompt engineering is cool and useful. https://arxiv.org/pdf/2310.15916 https://arxiv.org/pdf/2310.15916 https://huggingface.co/docs/peft/conceptual_guides/prompting https://huggingface.co/docs/peft/conceptual_guides/prompting

BoiledCabbage 10mo ago
> It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled Are you not aware the random sampling makes something non-deterministic?
- rTX5CMRXIfFG 10mo ago
  I don’t believe that when he wrote that, he was using his own intelligence
  - potatoman22 10mo ago
    At least I'm adding to the discussion.
    - rTX5CMRXIfFG 10mo ago
      People who disagree with you add to the discussion too
- potatoman22 10mo ago
  I'm saying LLMs are deterministic and because of that, prompt engineering can be effective. You knew what I was trying to say, but chose to ignore it. You should follow the HN Guidelines. I'm trying to have a discussion, not a snarkfest. > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.

theshrike79 10mo ago

You know what else is nondeterministic? "Project Management" You tell humans to make code, they make code. How do you check it's correct and does what it should?

abenga 10mo ago
Humans have agency and accountability, duty of care, etc.
- theshrike79 10mo ago
  How does a random programmer have "duty of care"? They're paid to ship a feature, why would they spend extra time making something perfect? If it breaks they just fix it - and get paid doing it. Not everyone writing code in a corporation is a Code Artesan who handcrafts every single character with thought and precision like an old Japanese craftsman. They just write code, it does about what it should and push a PR branch. Then you grab the next task from the corporate Jira project.
  - abenga 10mo ago
    > How does a random programmer have "duty of care"? Are there jobs that don't care if you don't take due care when writing code? I'd like me one of those.