7 ms·

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programmi

by ubj 3y ago

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

intelVISA 3y ago
Hear me out, just incubated a hot new lang that's about to capture the market and VC hearts: SELECT * FROM llm
- madmax108 3y ago
  I know you are probably joking, but: https://lmql.ai/ https://lmql.ai/
lcnPylGDnU4H9OF 3y ago
It won't necessarily turn into some that is fundamentally the same as a current programming language. Rather than a "VM" or "interpreter" or "compiler" we have this "LLM". Even if it requires a lot of domain knowledge to program using an "LLM-interpreted" language, the means of specification (in terms of how the software code is interpreted) may be different enough that it enables easier-to-write, more robust, (more Good Thing) etc. programs.
- davidthewatson 3y ago
  This is a hopeful evolutionary path. My concern is that I can literally feel Conway's law emanating from current LLM approaches as they switch between the actual LLM and the governing code around it that layers a buch of conditionals of the form: if (unspeakable_things): return negatory_good_buddy I see this happen a few times per day where the UI triggers a cancel even on its own fake typing mode and overwrites a user response that has at least half-rendered the trigger-warning-inducing response. It's pretty clear from a design perspective that this is intended to be proxy to facial expressions while being worthy of an MVP postmortem discussion about what viability means in a product that's somewhere on a spectrum of unintended consequences that only arise at runtime.
  - davidthewatson 3y ago
    This happened to me today on a prompt that I could not discern fit my original post as to "unspeakable things": * design a men's haircut by combining a 1/4" shaved undercut around the ears and neck with a longer 2" crown and intended to provide cover from the sun on top. followed by the AI interrupting itself mid-stream yet again after it had already answered the previous prompt to completion by providing step by step instructions to execute such a haircut. * I'm sorry, I can't respond to your prompt. Please try something else. My general impression is that there is near zero quality control oversight going on in this team and to their credit, that's been unusual in my experience observing and using M$ software post-Nadella.
felideon 3y ago
A number of years ago we were designing a way to specify insurance claim adjudication rules in natural language, so that "the business" could write their own rules. The "natural" language we ended up with was not so natural after all. We would have had to teach users this specific English dialect and grammar (formal and precise syntax, as you said). So, in the end, we abandoned that project and years later just rewrote the system so we could write claim rules in EDN format (from the Clojure world) to make our own lives easier. In theory, the business users could also learn how to write in this EDN format, but it wasn't something the stakeholders outside of engineering even wanted. On the one hand, their expertise was in insurance claims---they didn't want to write code. More importantly, they felt they would be held accountable for any mistakes in the rules that could well result in thousands and thousands of dollars in overpayments. Something the engineers weren't impervious to, but there's a good reason we have quality assurance measures.
- tomduncalf 3y ago
  > but it wasn't something the stakeholders outside of engineering even wanted Ha this reminds me of the craze for BDD/Cucumber type testing. Don’t think I ever once saw a product owner take interest in a human readable test case haha
  - jaggederest 3y ago
    I've used Cucumber on a few consulting projects I've done and had management / C-level interested and involved. It's a pretty narrow niche, but they were definitely enthusiastic for the idea that we had a defined list of features that we could print out (!!) as green or red for the current release. They had some previous negative experiences with uncertainty about what "was working" in releases, and a pretty slapdash process before I came on board, so it was an important trust building tool.
    - jamiek88 3y ago
      > important trust building tool This is so often completely missed in these conversations about these tools. Great point.
EarthLaunch 3y ago
Use LLM for the broad strokes, then fall back into 'hardcore JS' for areas that require guarantees or optimization. Like JS with fallback to C, and C with fallback to assembly. I like the idea.
oldagents 3y ago
Of course. Laws are the same way; defined norms for future behavior. The professional managerial class must maintain appropriate distinctions between their rights and ours. Their belief in exclusive right to profit from our agency is at risk if AI can generate too much noise.
jazzkingrt 3y ago
I think LLMs can transform between precise and imprecise languages. So it's useful to have a library that helps and the input or output be precise, when that is what the task involves.
TaylorAlexander 3y ago
Well to be fair, yes we do need to integrate programming languages with large neural nets in more advanced ways. I don’t think it’s really reinventing it so much as learning how to integrate these two different computing concepts.
eternalban 3y ago
So far it it reminds of the worst days of code embedded in templates. Once these things start getting into multipage prompts they will be hopelessly obscure. The second immediate thing that jumps out is 'fragility'. This will be the sort of codebase that original "prompt engineer" wrote and left and no one will touch it for fear of breaking humpty dumpty.
madrox 3y ago
The lovely thing about LLMs is that it can handle poorly worded prompts and well worded prompts. On the engineering side, we'll certainly see more rigor and best practices. For your average user? They can keep throwing whatever they like at it.
- jweir 3y ago
  Exactly. I have been using OpenAI for taking transcriptions and finding keywords/phrases that belong to particular categories. There are existing tools/services that do this – but I would need to learn their API. With OpenAI, I described it in English, provided sample JSON that I would like, run some tests, adjust and then I am ready. There was no manual to read, it is in my format, and the language is natural. And that is what I like about all this -- putting folks with limited technical skills in power.
  - andai 3y ago
    Have you used the OpenAI embeddings AI? It is used to find closely related pieces of text. You could split the target text into sentences or even words and run it through that. That'll be 5x cheaper (per token) than gpt-3.5-turbo and might be faster too, especially if you submit each word in parallel (asynchronously! Ask GPT for the code). The rate limits are per-token. Not sure if it's suitable for your use-case on its own, but it could at least work as a pre-filtering step if your costs are high. (The asynchronous speedup trick works for gpt-3 too of course.)
    - jweir 3y ago
      I have not yet played with embedding. It is on my list though. Fortunately for my current purposes 3.5-turbo is fast enough and quite affordable.
startupsfail 3y ago
We really need to start thinking of how to reduce magical thinking in the field. It’s not pretty. They literally quote biblical guidance for the models and pray that this would work. And start their prompts with “You”. Who is “You”?
- hxugufjfjf 3y ago
  The LLM. The most common end-user interface for LLM is a chat so the ser expects to be talking to someone or something.
- nomel 3y ago
  “You” is an optimization for the human user. Here’s some insight: https://news.ycombinator.com/item?id=35925154 https://news.ycombinator.com/item?id=35925154
  - startupsfail 3y ago
    If you see any prompt that starts with You, generally it is a poor design. Like using a “goto” or global variables.
    - nomel 3y ago
      This is true for something like raw GPT. For the chat models that have been specifically optimized for "you" prompts, this is false. See the discussion in the link I provided, along with the leaked copilot/bing prompts. Or, in other words, use a model in a way that fully takes advantage of how it was specifically optimized, from the intentional burning of massive amounts of compute time/money to get it that way.
      - startupsfail 3y ago
        Bing prompt is hilarious. And so wrong in so many ways.
        3y ago
deleted 3y ago
[deleted]
joe_the_user 3y ago
But is it a step to greater rigor? Or is it an illusion of rigor? They talk about improving tokenization but I don't believe that's the fundamental problem of controlling LLMs. The problem with LLMs is all the data comes in as (tokenized) language and the result is nothing but in-context predicted output. That's where all the "prompt-injection" exploits come from - as well as the hallucinations, "temper tantrums" and so-forth.
- startupsfail 3y ago
  It is not a step towards greater rigor. They literally have magical thinking and “biblical” quotes from GPT 11:4 all other the place, mixing code and religion. And starting prompts with “You”? Seriously. Can we at least drop that as a start?
  - quenix 3y ago
    > And starting prompts with “You”? Seriously. Can we at least drop that as a start? What is wrong with this?
    - startupsfail 3y ago
      “You” is completely unnecessary. What needs to be defined is the content of the language being modeled, not the model itself. And if there is an attempt to define the model itself, then this definition should be correct, should not contradict anything and should be useful. Otherwise it’s just dead code, waiting to create problems.
      - kurisufag 3y ago
        > Otherwise it’s just dead code, waiting to create problems it's very possible that the pretense improves results: most recorded interactions /are/ between two people, after all.
        dolmen 3y ago
conradev 3y ago
I don’t think formal languages are going anywhere because we need the guarantees that they can provide. From Dijkstra: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD667.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E... You need to be able to define all of the possible edge cases so there isn’t any Undefined Behavior: that’s the formal part Humans can use LLMs to manipulate these languages to achieve specific goals. I can imagine designing formal languages intended for LLMs to manipulate or generate, but I can’t imagine the need for the languages themselves going away.
- deleted 3y ago
  [deleted]
- DonaldPShimoda 3y ago
  > LLMs, like humans, can manipulate these languages Absolutely not. LLMs do not "manipulate" language. They do not have agency. They are extremely advanced text prediction engines. Their output is the result of applying the statistics harvested and distilled from existing uses of natural language. They only "appear" human because they are statistically geared toward producing human-like sequences of words. They cannot choose to change how they use language, and thus cannot be said to actively "manipulate" the language.
  - conradev 3y ago
    I made some edits, does that satisfy your constraints? Humans are the agent, LLMs the tool
  - jadefox 3y ago
    That “appearance” is pretty good at triggering our anthropomorphizing behaviors. I like your handle, did you read Richard Bach’s Illusions by any chance?
    - DonaldPShimoda 3y ago
      > That “appearance” is pretty good at triggering our anthropomorphizing behaviors. It's truly unfortunate, because I think that tendency for people to anthropomorphize LLMs leads to people thinking they can trust LLMs and take their words at face value, which is a path to significant problems, I think. > I like your handle, did you read Richard Bach’s Illusions by any chance? Absolutely, yes — good catch!
aristus 3y ago
Only partially tongue in cheek: have you tried asking it for an optimal syntax?
chaxor 3y ago
It's rigor applied where we don't need it, and ignores where we do (mathematical proofs and NN theory, architecture, hyper parameters, training schemes, etc). I have a somewhat irrational hatred towards almost all of the prompt oriented stuff being thrown about recently. There are a few (very few) input related training schemes that are interesting, but quite a bit of the "proompt-physicians" are just heralding the idea of essentially 'concise and effective communication' as 'I'm a ML expert now' ... which is annoying.
- Der_Einzige 3y ago
  Why would you dislike actual prompt engineering? This isn't some grifter trying to claim they're an expert because they wrote a cool prompt, this is a full fledged structured templating system for LLMs from an excellent author whose done a ton of other ML work. I think you should attack actual grifters instead of an excellent project.
chefandy 3y ago
Maybe someone will make an LLM with equivalent functionality to python that you can conveniently control with python syntax.

by ubj 3y ago

intelVISA 3y ago

Hear me out, just incubated a hot new lang that's about to capture the market and VC hearts: SELECT * FROM llm

madmax108 3y ago
I know you are probably joking, but: https://lmql.ai/ https://lmql.ai/

lcnPylGDnU4H9OF 3y ago

It won't necessarily turn into some that is fundamentally the same as a current programming language. Rather than a "VM" or "interpreter" or "compiler" we have this "LLM". Even if it requires a lot of domain knowledge to program using an "LLM-interpreted" language, the means of specification (in terms of how the software code is interpreted) may be different enough that it enables easier-to-write, more robust, (more Good Thing) etc. programs.

davidthewatson 3y ago
This is a hopeful evolutionary path. My concern is that I can literally feel Conway's law emanating from current LLM approaches as they switch between the actual LLM and the governing code around it that layers a buch of conditionals of the form: if (unspeakable_things): return negatory_good_buddy I see this happen a few times per day where the UI triggers a cancel even on its own fake typing mode and overwrites a user response that has at least half-rendered the trigger-warning-inducing response. It's pretty clear from a design perspective that this is intended to be proxy to facial expressions while being worthy of an MVP postmortem discussion about what viability means in a product that's somewhere on a spectrum of unintended consequences that only arise at runtime.
- davidthewatson 3y ago
  This happened to me today on a prompt that I could not discern fit my original post as to "unspeakable things": * design a men's haircut by combining a 1/4" shaved undercut around the ears and neck with a longer 2" crown and intended to provide cover from the sun on top. followed by the AI interrupting itself mid-stream yet again after it had already answered the previous prompt to completion by providing step by step instructions to execute such a haircut. * I'm sorry, I can't respond to your prompt. Please try something else. My general impression is that there is near zero quality control oversight going on in this team and to their credit, that's been unusual in my experience observing and using M$ software post-Nadella.

felideon 3y ago

A number of years ago we were designing a way to specify insurance claim adjudication rules in natural language, so that "the business" could write their own rules. The "natural" language we ended up with was not so natural after all. We would have had to teach users this specific English dialect and grammar (formal and precise syntax, as you said). So, in the end, we abandoned that project and years later just rewrote the system so we could write claim rules in EDN format (from the Clojure world) to make our own lives easier. In theory, the business users could also learn how to write in this EDN format, but it wasn't something the stakeholders outside of engineering even wanted. On the one hand, their expertise was in insurance claims---they didn't want to write code. More importantly, they felt they would be held accountable for any mistakes in the rules that could well result in thousands and thousands of dollars in overpayments. Something the engineers weren't impervious to, but there's a good reason we have quality assurance measures.

tomduncalf 3y ago
> but it wasn't something the stakeholders outside of engineering even wanted Ha this reminds me of the craze for BDD/Cucumber type testing. Don’t think I ever once saw a product owner take interest in a human readable test case haha
- jaggederest 3y ago
  I've used Cucumber on a few consulting projects I've done and had management / C-level interested and involved. It's a pretty narrow niche, but they were definitely enthusiastic for the idea that we had a defined list of features that we could print out (!!) as green or red for the current release. They had some previous negative experiences with uncertainty about what "was working" in releases, and a pretty slapdash process before I came on board, so it was an important trust building tool.
  - jamiek88 3y ago
    > important trust building tool This is so often completely missed in these conversations about these tools. Great point.

EarthLaunch 3y ago

Use LLM for the broad strokes, then fall back into 'hardcore JS' for areas that require guarantees or optimization. Like JS with fallback to C, and C with fallback to assembly. I like the idea.

oldagents 3y ago

Of course. Laws are the same way; defined norms for future behavior. The professional managerial class must maintain appropriate distinctions between their rights and ours. Their belief in exclusive right to profit from our agency is at risk if AI can generate too much noise.

jazzkingrt 3y ago

I think LLMs can transform between precise and imprecise languages. So it's useful to have a library that helps and the input or output be precise, when that is what the task involves.

TaylorAlexander 3y ago

Well to be fair, yes we do need to integrate programming languages with large neural nets in more advanced ways. I don’t think it’s really reinventing it so much as learning how to integrate these two different computing concepts.

eternalban 3y ago

So far it it reminds of the worst days of code embedded in templates. Once these things start getting into multipage prompts they will be hopelessly obscure. The second immediate thing that jumps out is 'fragility'. This will be the sort of codebase that original "prompt engineer" wrote and left and no one will touch it for fear of breaking humpty dumpty.

madrox 3y ago

The lovely thing about LLMs is that it can handle poorly worded prompts and well worded prompts. On the engineering side, we'll certainly see more rigor and best practices. For your average user? They can keep throwing whatever they like at it.

jweir 3y ago
Exactly. I have been using OpenAI for taking transcriptions and finding keywords/phrases that belong to particular categories. There are existing tools/services that do this – but I would need to learn their API. With OpenAI, I described it in English, provided sample JSON that I would like, run some tests, adjust and then I am ready. There was no manual to read, it is in my format, and the language is natural. And that is what I like about all this -- putting folks with limited technical skills in power.
- andai 3y ago
  Have you used the OpenAI embeddings AI? It is used to find closely related pieces of text. You could split the target text into sentences or even words and run it through that. That'll be 5x cheaper (per token) than gpt-3.5-turbo and might be faster too, especially if you submit each word in parallel (asynchronously! Ask GPT for the code). The rate limits are per-token. Not sure if it's suitable for your use-case on its own, but it could at least work as a pre-filtering step if your costs are high. (The asynchronous speedup trick works for gpt-3 too of course.)
  - jweir 3y ago
    I have not yet played with embedding. It is on my list though. Fortunately for my current purposes 3.5-turbo is fast enough and quite affordable.

startupsfail 3y ago

We really need to start thinking of how to reduce magical thinking in the field. It’s not pretty. They literally quote biblical guidance for the models and pray that this would work. And start their prompts with “You”. Who is “You”?

hxugufjfjf 3y ago
The LLM. The most common end-user interface for LLM is a chat so the ser expects to be talking to someone or something.
nomel 3y ago
“You” is an optimization for the human user. Here’s some insight: https://news.ycombinator.com/item?id=35925154 https://news.ycombinator.com/item?id=35925154
- startupsfail 3y ago
  If you see any prompt that starts with You, generally it is a poor design. Like using a “goto” or global variables.
  - nomel 3y ago
    This is true for something like raw GPT. For the chat models that have been specifically optimized for "you" prompts, this is false. See the discussion in the link I provided, along with the leaked copilot/bing prompts. Or, in other words, use a model in a way that fully takes advantage of how it was specifically optimized, from the intentional burning of massive amounts of compute time/money to get it that way.
    - startupsfail 3y ago
      Bing prompt is hilarious. And so wrong in so many ways.
      - 3y ago

deleted 3y ago

[deleted]

joe_the_user 3y ago

But is it a step to greater rigor? Or is it an illusion of rigor? They talk about improving tokenization but I don't believe that's the fundamental problem of controlling LLMs. The problem with LLMs is all the data comes in as (tokenized) language and the result is nothing but in-context predicted output. That's where all the "prompt-injection" exploits come from - as well as the hallucinations, "temper tantrums" and so-forth.

startupsfail 3y ago
It is not a step towards greater rigor. They literally have magical thinking and “biblical” quotes from GPT 11:4 all other the place, mixing code and religion. And starting prompts with “You”? Seriously. Can we at least drop that as a start?
- quenix 3y ago
  > And starting prompts with “You”? Seriously. Can we at least drop that as a start? What is wrong with this?
  - startupsfail 3y ago
    “You” is completely unnecessary. What needs to be defined is the content of the language being modeled, not the model itself. And if there is an attempt to define the model itself, then this definition should be correct, should not contradict anything and should be useful. Otherwise it’s just dead code, waiting to create problems.
    - kurisufag 3y ago
      > Otherwise it’s just dead code, waiting to create problems it's very possible that the pretense improves results: most recorded interactions /are/ between two people, after all.
      - dolmen 3y ago

conradev 3y ago

I don’t think formal languages are going anywhere because we need the guarantees that they can provide. From Dijkstra: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD667.html https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E... You need to be able to define all of the possible edge cases so there isn’t any Undefined Behavior: that’s the formal part Humans can use LLMs to manipulate these languages to achieve specific goals. I can imagine designing formal languages intended for LLMs to manipulate or generate, but I can’t imagine the need for the languages themselves going away.

deleted 3y ago
[deleted]
DonaldPShimoda 3y ago
> LLMs, like humans, can manipulate these languages Absolutely not. LLMs do not "manipulate" language. They do not have agency. They are extremely advanced text prediction engines. Their output is the result of applying the statistics harvested and distilled from existing uses of natural language. They only "appear" human because they are statistically geared toward producing human-like sequences of words. They cannot choose to change how they use language, and thus cannot be said to actively "manipulate" the language.
- conradev 3y ago
  I made some edits, does that satisfy your constraints? Humans are the agent, LLMs the tool
- jadefox 3y ago
  That “appearance” is pretty good at triggering our anthropomorphizing behaviors. I like your handle, did you read Richard Bach’s Illusions by any chance?
  - DonaldPShimoda 3y ago
    > That “appearance” is pretty good at triggering our anthropomorphizing behaviors. It's truly unfortunate, because I think that tendency for people to anthropomorphize LLMs leads to people thinking they can trust LLMs and take their words at face value, which is a path to significant problems, I think. > I like your handle, did you read Richard Bach’s Illusions by any chance? Absolutely, yes — good catch!

aristus 3y ago

Only partially tongue in cheek: have you tried asking it for an optimal syntax?

chaxor 3y ago

It's rigor applied where we don't need it, and ignores where we do (mathematical proofs and NN theory, architecture, hyper parameters, training schemes, etc). I have a somewhat irrational hatred towards almost all of the prompt oriented stuff being thrown about recently. There are a few (very few) input related training schemes that are interesting, but quite a bit of the "proompt-physicians" are just heralding the idea of essentially 'concise and effective communication' as 'I'm a ML expert now' ... which is annoying.

Der_Einzige 3y ago
Why would you dislike actual prompt engineering? This isn't some grifter trying to claim they're an expert because they wrote a cool prompt, this is a full fledged structured templating system for LLMs from an excellent author whose done a ton of other ML work. I think you should attack actual grifters instead of an excellent project.

chefandy 3y ago

Maybe someone will make an LLM with equivalent functionality to python that you can conveniently control with python syntax.